[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:04 dist-test-slave-gkw7 krb5kdc[11732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:04 dist-test-slave-gkw7 krb5kdc[11732](info): set up 2 sockets
Aug 07 06:38:04 dist-test-slave-gkw7 krb5kdc[11732](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 06:38:05.589627 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:05 dist-test-slave-gkw7 krb5kdc[11732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:05.723093 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:46147
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:46147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:05.810184 11750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:05.810387 11750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:05.810410 11750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:05.811518 11750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:05.811555 11750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:05.811576 11750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:05.811589 11750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:05.811601 11750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:05.813217 11750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:46147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:46147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:05.813638 11750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:05.813879 11750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:05.820683 11756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:05.820663 11755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:05.822104 11750 server_base.cc:1034] running on GCE node
W20260807 06:38:05.822448 11758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:05.822780 11750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:05.823300 11750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:05.824456 11750 hybrid_clock.cc:648] HybridClock initialized: now 1786084685824429 us; error 42 us; skew 500 ppm
Aug 07 06:38:05 dist-test-slave-gkw7 krb5kdc[11732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084685, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:05.832957 11750 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:05.833766 11750 webserver.cc:460] Webserver started at http://127.11.114.254:34631/ using document root <none> and password file <none>
I20260807 06:38:05.834124 11750 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:05.834206 11750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:05.834400 11750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:05.835712 11750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "a5a6aeb562c544bc93560304c1ebe6b4"
format_stamp: "Formatted at 2026-08-07 06:38:05 on dist-test-slave-gkw7"
server_key: "d0528f5ce76bdf1505d09d6ef2b8d1ba"
server_key_iv: "d97b6f84c33ef86ebceb95820acc956e"
server_key_version: "encryptionkey@0"
I20260807 06:38:05.836108 11750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "a5a6aeb562c544bc93560304c1ebe6b4"
format_stamp: "Formatted at 2026-08-07 06:38:05 on dist-test-slave-gkw7"
server_key: "d0528f5ce76bdf1505d09d6ef2b8d1ba"
server_key_iv: "d97b6f84c33ef86ebceb95820acc956e"
server_key_version: "encryptionkey@0"
I20260807 06:38:05.838069 11750 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 06:38:05.839069 11767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:05.839354 11750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:05.839479 11750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "a5a6aeb562c544bc93560304c1ebe6b4"
format_stamp: "Formatted at 2026-08-07 06:38:05 on dist-test-slave-gkw7"
server_key: "d0528f5ce76bdf1505d09d6ef2b8d1ba"
server_key_iv: "d97b6f84c33ef86ebceb95820acc956e"
server_key_version: "encryptionkey@0"
I20260807 06:38:05.839555 11750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:05.854143 11750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:05.856436 11750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:05.856652 11750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:05.862244 11750 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:46147
I20260807 06:38:05.862309 11847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:46147 every 8 connection(s)
I20260807 06:38:05.862943 11750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:05.864461 11849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:05.867671 11849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4: Bootstrap starting.
I20260807 06:38:05.868621 11849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:05.868949 11849 log.cc:824] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:05.869431 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 11750
I20260807 06:38:05.869518 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:05.870000 11849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4: No bootstrap required, opened a new log
I20260807 06:38:05.870245 11723 external_mini_cluster.cc:1428] Setting key fa78a576cd41f53f2ffab744d892fb90
I20260807 06:38:05.872256 11849 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a6aeb562c544bc93560304c1ebe6b4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46147 } }
I20260807 06:38:05.872473 11849 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:05.872596 11849 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5a6aeb562c544bc93560304c1ebe6b4, State: Initialized, Role: FOLLOWER
I20260807 06:38:05.872810 11849 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a6aeb562c544bc93560304c1ebe6b4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46147 } }
I20260807 06:38:05.872974 11849 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:05.873023 11849 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:05.873075 11849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:05.874348 11849 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a6aeb562c544bc93560304c1ebe6b4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46147 } }
I20260807 06:38:05.874572 11849 leader_election.cc:302] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5a6aeb562c544bc93560304c1ebe6b4; no voters:
I20260807 06:38:05.874801 11849 leader_election.cc:288] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:05.874856 11856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:05.875090 11856 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [term 1 LEADER]: Becoming Leader. State: Replica: a5a6aeb562c544bc93560304c1ebe6b4, State: Running, Role: LEADER
I20260807 06:38:05.875291 11856 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a6aeb562c544bc93560304c1ebe6b4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46147 } }
I20260807 06:38:05.875334 11849 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:05.875907 11856 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5a6aeb562c544bc93560304c1ebe6b4. Latest consensus state: current_term: 1 leader_uuid: "a5a6aeb562c544bc93560304c1ebe6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a6aeb562c544bc93560304c1ebe6b4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46147 } } }
I20260807 06:38:05.876399 11856 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:05.876314 11858 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5a6aeb562c544bc93560304c1ebe6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a6aeb562c544bc93560304c1ebe6b4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46147 } } }
I20260807 06:38:05.876936 11858 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4 [sys.catalog]: This master's current role is: LEADER
W20260807 06:38:05.878106 11873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:38:05.878180 11873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:05.878408 11867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:05.878717 11867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:05.880620 11867 catalog_manager.cc:1347] Generated new cluster ID: f9e33b0a047f4f318c4cb4f4281772b0
I20260807 06:38:05.880677 11867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:38:05 dist-test-slave-gkw7 krb5kdc[11732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:05.890877 11855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:05.871522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.11.114.254:46147)
0807 06:38:05.871880 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:05.871890 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:05.872036 (+ 146us) server_negotiation.cc:407] Connection header received
0807 06:38:05.874122 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:05.874130 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:05.874241 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:05.874414 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:05.876976 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:05.878361 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:05.880282 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:05.880985 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:05.881097 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:05.886039 (+ 4942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:05.886045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:05.886045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:05.886059 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:05.888441 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:05.888991 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:05.888995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:05.888996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:05.889153 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:05.889467 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:05.889468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:05.889468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:05.889864 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:05.890060 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:05.890350 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 06:38:05.890575 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":154,"threads_started":1}
I20260807 06:38:05.897224 11867 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:05.897897 11867 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:05.903427 11867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5a6aeb562c544bc93560304c1ebe6b4: Generated new TSK 0
I20260807 06:38:05.903644 11867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:05.944998 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46147
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:06.031183 11884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:06.031400 11884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:06.031426 11884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:06.032599 11884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:06.032644 11884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:06.032672 11884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:06.034220 11884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46147
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:06.034507 11884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:06.034763 11884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:06.040359 11889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:06.040578 11884 server_base.cc:1034] running on GCE node
W20260807 06:38:06.040401 11893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:06.040376 11890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:06.040964 11884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:06.041250 11884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:06.042438 11884 hybrid_clock.cc:648] HybridClock initialized: now 1786084686042418 us; error 38 us; skew 500 ppm
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:06.045745 11884 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:06.046357 11884 webserver.cc:460] Webserver started at http://127.11.114.193:41097/ using document root <none> and password file <none>
I20260807 06:38:06.046618 11884 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:06.046695 11884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:06.046865 11884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:06.048043 11884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "5426782ad4114845b7e44602c75a1402"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "dc0e1142ff8c8ccb642d2d06d42915d1"
server_key_iv: "47c60094edaeafdc98316d7f8136d7fc"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.048462 11884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "5426782ad4114845b7e44602c75a1402"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "dc0e1142ff8c8ccb642d2d06d42915d1"
server_key_iv: "47c60094edaeafdc98316d7f8136d7fc"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.050120 11884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:06.051266 11903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.051512 11884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:06.051656 11884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "5426782ad4114845b7e44602c75a1402"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "dc0e1142ff8c8ccb642d2d06d42915d1"
server_key_iv: "47c60094edaeafdc98316d7f8136d7fc"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.051744 11884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:06.072548 11884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:06.075416 11884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:06.075656 11884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:06.076027 11884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:06.076467 11884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:06.076529 11884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.076556 11884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:06.076570 11884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.084883 11884 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:43339
I20260807 06:38:06.085000 12064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:43339 every 8 connection(s)
I20260807 06:38:06.085390 11884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:06.091411 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 11884
I20260807 06:38:06.091521 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:06.091694 11723 external_mini_cluster.cc:1428] Setting key f6243b68d5a6a6e14e07072cfe033ffb
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:06.096552 11855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:06.086480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:36335 (local address 127.11.114.254:46147)
0807 06:38:06.086559 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:06.086561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:06.086801 (+ 240us) server_negotiation.cc:407] Connection header received
0807 06:38:06.087379 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:06.087380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:06.087395 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:06.087446 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:06.088658 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.089183 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.090247 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.090459 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.090577 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:06.093163 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:06.093165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:06.093165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:06.093173 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:06.094968 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.095420 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.095421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.095422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.095496 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.095757 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.095758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.095758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.095915 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:06.095990 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:06.096279 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 06:38:06.096395 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:38:06.096632 12068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:06.086444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46147 (local address 127.11.114.193:36335)
0807 06:38:06.086670 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:06.086709 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:06.087264 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:06.087536 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:06.087540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:06.088344 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:06.088548 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:06.088553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.089299 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.089300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:06.090152 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:06.090154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.090587 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.090588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:06.090720 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:06.091332 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:06.091343 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:06.093064 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:06.095067 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:06.095075 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:06.095077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:06.095335 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:06.095570 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:06.095571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:06.095571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:06.095681 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:06.095992 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:06.095997 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:06.096136 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 06:38:06.096234 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:38:06.096369 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":105,"threads_started":1}
I20260807 06:38:06.097354 12066 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46147
I20260807 06:38:06.097472 12066 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:06.097719 12066 heartbeater.cc:511] Master 127.11.114.254:46147 requested a full tablet report, sending...
I20260807 06:38:06.098709 11789 ts_manager.cc:194] Registered new tserver with Master: 5426782ad4114845b7e44602c75a1402 (127.11.114.193:43339)
I20260807 06:38:06.099483 11789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:36335
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:06.120771 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46147
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:06.211208 12073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:06.211429 12073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:06.211453 12073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:06.212533 12073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:06.212572 12073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:06.212598 12073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:06.213994 12073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46147
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:06.214174 12073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:06.214368 12073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:06.219543 12084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:06.219591 12083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:06.219666 12086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:06.219717 12073 server_base.cc:1034] running on GCE node
I20260807 06:38:06.219973 12073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:06.220216 12073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:06.221411 12073 hybrid_clock.cc:648] HybridClock initialized: now 1786084686221394 us; error 29 us; skew 500 ppm
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:06.224057 12073 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:06.224503 12073 webserver.cc:460] Webserver started at http://127.11.114.194:39473/ using document root <none> and password file <none>
I20260807 06:38:06.224743 12073 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:06.224812 12073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:06.224970 12073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:06.226203 12073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "50874f6648c64f288573b662dc787c4c"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "a9306bbdeda8e5aae1b98170ab2ccac8"
server_key_iv: "0b259901902db8766e78334dc5d15059"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.226655 12073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "50874f6648c64f288573b662dc787c4c"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "a9306bbdeda8e5aae1b98170ab2ccac8"
server_key_iv: "0b259901902db8766e78334dc5d15059"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.228029 12073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:06.228803 12095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.228989 12073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:06.229096 12073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "50874f6648c64f288573b662dc787c4c"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "a9306bbdeda8e5aae1b98170ab2ccac8"
server_key_iv: "0b259901902db8766e78334dc5d15059"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.229173 12073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:06.254249 12073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:06.256500 12073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:06.256744 12073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:06.258594 12073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:06.258960 12073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:06.259017 12073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.259052 12073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:06.259081 12073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.266219 12073 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:34985
I20260807 06:38:06.266263 12261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:34985 every 8 connection(s)
I20260807 06:38:06.266649 12073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:06.267001 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 12073
I20260807 06:38:06.267099 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:06.267292 11723 external_mini_cluster.cc:1428] Setting key 831a4197c782cf80cb93ab5a8106e0e2
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
I20260807 06:38:06.278376 11855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:06.267627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:47389 (local address 127.11.114.254:46147)
0807 06:38:06.267678 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:06.267681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:06.267935 (+ 254us) server_negotiation.cc:407] Connection header received
0807 06:38:06.268535 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:06.268536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:06.268549 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:06.268619 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:06.269757 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.270366 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.271705 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.272050 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.272126 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:06.274721 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:06.274723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:06.274724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:06.274732 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:06.276690 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.277140 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.277141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.277142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.277274 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.277642 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.277644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.277644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.277780 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:06.277871 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:06.278228 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 06:38:06.278304 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:38:06.278622 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:06.267591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46147 (local address 127.11.114.194:47389)
0807 06:38:06.267791 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:06.267836 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:06.268430 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:06.268681 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:06.268684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:06.269354 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:06.269652 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:06.269664 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.270477 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.270478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:06.271609 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:06.271613 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.272150 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.272151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:06.272275 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:06.273034 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:06.273043 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:06.274621 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:06.276777 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:06.276784 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:06.276786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:06.277025 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:06.277360 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:06.277361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:06.277362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:06.277485 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:06.277860 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:06.277868 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:06.278017 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 06:38:06.278190 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 06:38:06.278361 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":105,"threads_started":1}
I20260807 06:38:06.279168 12263 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46147
I20260807 06:38:06.279282 12263 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:06.279474 12263 heartbeater.cc:511] Master 127.11.114.254:46147 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
I20260807 06:38:06.280028 11789 ts_manager.cc:194] Registered new tserver with Master: 50874f6648c64f288573b662dc787c4c (127.11.114.194:34985)
I20260807 06:38:06.280514 11789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:47389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:06.295763 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46147
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:06.386883 12270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:06.387135 12270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:06.387178 12270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:06.388428 12270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:06.388502 12270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:06.388550 12270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:06.390183 12270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46147
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:06.390461 12270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:06.390702 12270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:06.396044 12278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:06.396138 12270 server_base.cc:1034] running on GCE node
W20260807 06:38:06.396054 12281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:06.396008 12277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:06.396436 12270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:06.396690 12270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:06.397853 12270 hybrid_clock.cc:648] HybridClock initialized: now 1786084686397840 us; error 27 us; skew 500 ppm
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:06.400368 12270 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:06.400862 12270 webserver.cc:460] Webserver started at http://127.11.114.195:39555/ using document root <none> and password file <none>
I20260807 06:38:06.401117 12270 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:06.401187 12270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:06.401343 12270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:06.402515 12270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "3a7db2a8e8cb4469b7426be2b470afc2"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "780359b78628a924a7bb6d6687fbf7d4"
server_key_iv: "c768c5a6ef141b8f9ae799e08cb09d6b"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.402894 12270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "3a7db2a8e8cb4469b7426be2b470afc2"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "780359b78628a924a7bb6d6687fbf7d4"
server_key_iv: "c768c5a6ef141b8f9ae799e08cb09d6b"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.404525 12270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:06.405248 12288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.405417 12270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:06.405509 12270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "3a7db2a8e8cb4469b7426be2b470afc2"
format_stamp: "Formatted at 2026-08-07 06:38:06 on dist-test-slave-gkw7"
server_key: "780359b78628a924a7bb6d6687fbf7d4"
server_key_iv: "c768c5a6ef141b8f9ae799e08cb09d6b"
server_key_version: "encryptionkey@0"
I20260807 06:38:06.405648 12270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:06.416860 12270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:06.418864 12270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:06.418973 12270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:06.419174 12270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:06.419538 12270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:06.419598 12270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.419641 12270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:06.419667 12270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:06.427067 12270 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:37731
I20260807 06:38:06.427103 12448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:37731 every 8 connection(s)
I20260807 06:38:06.427455 12270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:38:06.432754 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 12270
I20260807 06:38:06.432866 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:06.433039 11723 external_mini_cluster.cc:1428] Setting key 5229739dac02830e8d91474cadd1ddfe
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:06.437716 11855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:06.428365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:37671 (local address 127.11.114.254:46147)
0807 06:38:06.428435 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:06.428437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:06.428664 (+ 227us) server_negotiation.cc:407] Connection header received
0807 06:38:06.429206 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:06.429208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:06.429222 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:06.429281 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:06.430315 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.430842 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.431885 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.432167 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.432244 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:06.435031 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:06.435033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:06.435033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:06.435040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:06.436316 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.436698 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.436699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.436699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.436754 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.437023 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.437025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.437025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.437152 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:06.437243 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:06.437513 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 06:38:06.437615 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:38:06.437942 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:06.428349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46147 (local address 127.11.114.195:37671)
0807 06:38:06.428525 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:06.428563 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:06.429094 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:06.429353 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:06.429357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:06.429990 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:06.430204 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:06.430210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.430940 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.430941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:06.431784 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:06.431787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.432255 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.432257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:06.432389 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:06.432943 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:06.432954 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:06.434950 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:06.436393 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:06.436400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:06.436401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:06.436634 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:06.436827 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:06.436828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:06.436828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:06.436954 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:06.437219 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:06.437225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:06.437378 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 06:38:06.437491 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 06:38:06.437684 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260807 06:38:06.438531 12449 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46147
I20260807 06:38:06.438635 12449 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:06.438818 12449 heartbeater.cc:511] Master 127.11.114.254:46147 requested a full tablet report, sending...
I20260807 06:38:06.439175 11789 ts_manager.cc:194] Registered new tserver with Master: 3a7db2a8e8cb4469b7426be2b470afc2 (127.11.114.195:37731)
I20260807 06:38:06.439570 11789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:37671
I20260807 06:38:06.445371 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:06.471850 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:06.463978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41134 (local address 127.11.114.193:43339)
0807 06:38:06.464173 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:06.464178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:06.464188 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:06.464217 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:06.464219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:06.464234 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:06.464312 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:06.464900 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.465746 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.466502 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.466829 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.466887 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:06.469069 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:06.469078 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:06.469078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:06.469085 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:06.470772 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.471087 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.471090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.471090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.471173 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.471403 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.471403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.471403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.471534 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:06.471644 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:06.471707 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 06:38:06.471769 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
W20260807 06:38:06.472393 11944 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41134
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[11732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:06.496307 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:06.489340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.11.114.193:43339)
0807 06:38:06.489410 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:06.489412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:06.489522 (+ 110us) server_negotiation.cc:407] Connection header received
0807 06:38:06.489623 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:06.489624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:06.489638 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:06.489690 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:06.490268 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.490764 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.491471 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:06.491667 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:06.491716 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:06.493940 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:06.493941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:06.493942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:06.493948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:06.495376 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.495666 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.495666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.495667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.495717 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:06.495860 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:06.495861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:06.495861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:06.495942 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:06.496023 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:06.496087 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 06:38:06.496172 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:38:06.496901 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 11884
I20260807 06:38:06.511829 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 12073
I20260807 06:38:06.522713 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 12270
I20260807 06:38:06.528729 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 11750
2026-08-07T06:38:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2184 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[12467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[12467](info): set up 2 sockets
Aug 07 06:38:06 dist-test-slave-gkw7 krb5kdc[12467](info): commencing operation
krb5kdc: starting...
W20260807 06:38:07.752802 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:07 dist-test-slave-gkw7 krb5kdc[12467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:07.877676 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33461
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:07.959432 12485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:07.959686 12485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:07.959725 12485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:07.960876 12485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:07.960952 12485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:07.960976 12485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:07.960999 12485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:07.961028 12485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:07.962533 12485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:33461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:07.962811 12485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:07.963047 12485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:07.968099 12496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:07.968148 12493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:07.968103 12492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:07.968328 12485 server_base.cc:1034] running on GCE node
I20260807 06:38:07.968570 12485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:07.968909 12485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:07.970054 12485 hybrid_clock.cc:648] HybridClock initialized: now 1786084687970025 us; error 45 us; skew 500 ppm
Aug 07 06:38:07 dist-test-slave-gkw7 krb5kdc[12467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084687, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:07.972693 12485 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:07.973141 12485 webserver.cc:460] Webserver started at http://127.11.114.254:42993/ using document root <none> and password file <none>
I20260807 06:38:07.973328 12485 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:07.973368 12485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:07.973459 12485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:07.974531 12485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "0b66dfebf5f4465cbed68d07f67e740b"
format_stamp: "Formatted at 2026-08-07 06:38:07 on dist-test-slave-gkw7"
server_key: "226f9e3ffce50391282b8d3834b5d9b6"
server_key_iv: "193ffbf923d6c9732e9d9906103e533b"
server_key_version: "encryptionkey@0"
I20260807 06:38:07.974849 12485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "0b66dfebf5f4465cbed68d07f67e740b"
format_stamp: "Formatted at 2026-08-07 06:38:07 on dist-test-slave-gkw7"
server_key: "226f9e3ffce50391282b8d3834b5d9b6"
server_key_iv: "193ffbf923d6c9732e9d9906103e533b"
server_key_version: "encryptionkey@0"
I20260807 06:38:07.976100 12485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:38:07.976824 12508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:07.977123 12485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:07.977197 12485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "0b66dfebf5f4465cbed68d07f67e740b"
format_stamp: "Formatted at 2026-08-07 06:38:07 on dist-test-slave-gkw7"
server_key: "226f9e3ffce50391282b8d3834b5d9b6"
server_key_iv: "193ffbf923d6c9732e9d9906103e533b"
server_key_version: "encryptionkey@0"
I20260807 06:38:07.977291 12485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:07.997444 12485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:08.000100 12485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:08.000276 12485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:08.004516 12485 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:33461
I20260807 06:38:08.004577 12575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:33461 every 8 connection(s)
I20260807 06:38:08.004945 12485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:08.005867 12576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:08.008167 12576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b: Bootstrap starting.
I20260807 06:38:08.008786 12576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:08.009093 12576 log.cc:824] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:08.009918 12576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b: No bootstrap required, opened a new log
I20260807 06:38:08.011525 12576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b66dfebf5f4465cbed68d07f67e740b" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33461 } }
I20260807 06:38:08.011667 12576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:08.011689 12576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b66dfebf5f4465cbed68d07f67e740b, State: Initialized, Role: FOLLOWER
I20260807 06:38:08.011870 12576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b66dfebf5f4465cbed68d07f67e740b" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33461 } }
I20260807 06:38:08.011945 12576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:08.011967 12576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:08.012042 12576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:08.012575 12576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b66dfebf5f4465cbed68d07f67e740b" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33461 } }
I20260807 06:38:08.012713 12576 leader_election.cc:302] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b66dfebf5f4465cbed68d07f67e740b; no voters:
I20260807 06:38:08.012785 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 12485
I20260807 06:38:08.012916 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:08.012946 12576 leader_election.cc:288] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:08.013041 12582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:08.013090 11723 external_mini_cluster.cc:1428] Setting key 0845b415d6cf29bb0201a7121e9ff39c
I20260807 06:38:08.013176 12582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [term 1 LEADER]: Becoming Leader. State: Replica: 0b66dfebf5f4465cbed68d07f67e740b, State: Running, Role: LEADER
I20260807 06:38:08.013262 12576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:08.013310 12582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b66dfebf5f4465cbed68d07f67e740b" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33461 } }
I20260807 06:38:08.013651 12582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b66dfebf5f4465cbed68d07f67e740b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b66dfebf5f4465cbed68d07f67e740b" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33461 } } }
I20260807 06:38:08.013736 12582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:08.013863 12582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b66dfebf5f4465cbed68d07f67e740b. Latest consensus state: current_term: 1 leader_uuid: "0b66dfebf5f4465cbed68d07f67e740b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b66dfebf5f4465cbed68d07f67e740b" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33461 } } }
I20260807 06:38:08.013931 12582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:08.014043 12592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:08.015007 12592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:08.016588 12592 catalog_manager.cc:1347] Generated new cluster ID: 908a4048bd034baeb1d935bb99ac04b9
I20260807 06:38:08.016660 12592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:08.031630 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.013626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.11.114.254:33461)
0807 06:38:08.014762 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.014772 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.014805 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:38:08.014923 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.014930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.014987 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.016134 (+ 1147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:08.016323 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.022935 (+ 6612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.024127 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.024798 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.024914 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.027863 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:08.027868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:08.027868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:08.027883 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:08.029930 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.030339 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.030342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.030342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.030463 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.030675 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.030675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.030676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.030979 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:08.031113 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.031247 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.031389 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1083,"thread_start_us":97,"threads_started":1}
I20260807 06:38:08.035033 12592 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:08.035563 12592 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:08.042616 12592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b66dfebf5f4465cbed68d07f67e740b: Generated new TSK 0
I20260807 06:38:08.042831 12592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:08.077072 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33461
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:08.163777 12612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:08.164024 12612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:08.164067 12612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:08.165267 12612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:08.165337 12612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:08.165383 12612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:08.166981 12612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33461
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:08.167269 12612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:08.167502 12612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:08.172749 12621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:08.172744 12623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:08.172744 12620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:08.173118 12612 server_base.cc:1034] running on GCE node
I20260807 06:38:08.173454 12612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:08.173727 12612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:08.174901 12612 hybrid_clock.cc:648] HybridClock initialized: now 1786084688174878 us; error 35 us; skew 500 ppm
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:08.177426 12612 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:08.177976 12612 webserver.cc:460] Webserver started at http://127.11.114.193:46101/ using document root <none> and password file <none>
I20260807 06:38:08.178267 12612 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:08.178345 12612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:08.178510 12612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:08.179610 12612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "40243b43802d41d4bcc82241d72fcf46"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "4addd0e5901e5745e890ff646c82e7f4"
server_key_iv: "045a51d1669f93220dae31400f9bcb5e"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.180007 12612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "40243b43802d41d4bcc82241d72fcf46"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "4addd0e5901e5745e890ff646c82e7f4"
server_key_iv: "045a51d1669f93220dae31400f9bcb5e"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.181455 12612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:08.182333 12633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.182557 12612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:08.182649 12612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "40243b43802d41d4bcc82241d72fcf46"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "4addd0e5901e5745e890ff646c82e7f4"
server_key_iv: "045a51d1669f93220dae31400f9bcb5e"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.182713 12612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:08.193495 12612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:08.195813 12612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:08.195925 12612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:08.196095 12612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:08.196424 12612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:08.196537 12612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.196599 12612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:08.196624 12612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.202530 12612 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:42255
I20260807 06:38:08.202589 12795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:42255 every 8 connection(s)
I20260807 06:38:08.202948 12612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:08.212525 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.203720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:53477 (local address 127.11.114.254:33461)
0807 06:38:08.203800 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.203803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.203970 (+ 167us) server_negotiation.cc:407] Connection header received
0807 06:38:08.204978 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.204979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.204994 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.205049 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:08.205799 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.206392 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.207447 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.207647 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.207699 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.210049 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:08.210050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:08.210051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:08.210056 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:08.211275 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.211632 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.211633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.211633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.211684 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.211944 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.211946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.211946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.212073 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:08.212141 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.212377 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.212456 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:38:08.212659 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.203657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33461 (local address 127.11.114.193:53477)
0807 06:38:08.203852 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:08.203888 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:08.204884 (+ 996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:08.205130 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:08.205133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:08.205430 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:08.205709 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.205715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.206498 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.206499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.207361 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.207364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.207762 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.207764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.207883 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.208494 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:08.208503 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:08.209961 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:08.211346 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:08.211353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:08.211355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:08.211568 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:08.211751 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:08.211751 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:08.211752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:08.211851 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:08.212140 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:08.212144 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:08.212259 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 06:38:08.212343 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 06:38:08.212464 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260807 06:38:08.212989 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 12612
I20260807 06:38:08.213090 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:08.213311 11723 external_mini_cluster.cc:1428] Setting key 60f7facfba347d6fc2bad54e46a8cdde
I20260807 06:38:08.213300 12796 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33461
I20260807 06:38:08.213398 12796 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:08.213627 12796 heartbeater.cc:511] Master 127.11.114.254:33461 requested a full tablet report, sending...
I20260807 06:38:08.214287 12532 ts_manager.cc:194] Registered new tserver with Master: 40243b43802d41d4bcc82241d72fcf46 (127.11.114.193:42255)
I20260807 06:38:08.214869 12532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:53477
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:08.240268 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33461
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:08.334935 12802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:08.335143 12802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:08.335167 12802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:08.336331 12802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:08.336371 12802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:08.336396 12802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:08.337890 12802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33461
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:08.338071 12802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:08.338255 12802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:08.343425 12809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:08.343451 12814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:08.343449 12812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:08.343560 12802 server_base.cc:1034] running on GCE node
I20260807 06:38:08.343875 12802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:08.344129 12802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:08.345274 12802 hybrid_clock.cc:648] HybridClock initialized: now 1786084688345273 us; error 50 us; skew 500 ppm
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:08.347796 12802 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:08.348310 12802 webserver.cc:460] Webserver started at http://127.11.114.194:34267/ using document root <none> and password file <none>
I20260807 06:38:08.348549 12802 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:08.348623 12802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:08.348759 12802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:08.349907 12802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "3f940789f3634797a8cd4b326ce09244"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "66b83bc110345756886035c940e0c82e"
server_key_iv: "5b95b87017038e0217f738ddc3269fd0"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.350284 12802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "3f940789f3634797a8cd4b326ce09244"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "66b83bc110345756886035c940e0c82e"
server_key_iv: "5b95b87017038e0217f738ddc3269fd0"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.351650 12802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:08.352363 12824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.352521 12802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:08.352624 12802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "3f940789f3634797a8cd4b326ce09244"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "66b83bc110345756886035c940e0c82e"
server_key_iv: "5b95b87017038e0217f738ddc3269fd0"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.352707 12802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:08.362205 12802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:08.363982 12802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:08.364153 12802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:08.364390 12802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:08.364735 12802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:08.364795 12802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.364867 12802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:08.364898 12802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.370905 12802 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:39467
I20260807 06:38:08.370986 12975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:39467 every 8 connection(s)
I20260807 06:38:08.371312 12802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:08.376307 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 12802
I20260807 06:38:08.376408 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:08.376596 11723 external_mini_cluster.cc:1428] Setting key 4c9211eb3a1e7d7ca24a1fe36acae204
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:08.382170 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.372098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:34987 (local address 127.11.114.254:33461)
0807 06:38:08.372166 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.372168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.372389 (+ 221us) server_negotiation.cc:407] Connection header received
0807 06:38:08.373294 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.373295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.373304 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.373362 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:08.374111 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.374608 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.375591 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.375778 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.375828 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.378727 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:08.378730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:08.378730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:08.378738 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:08.380534 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.380986 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.380987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.380988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.381063 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.381370 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.381371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.381371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.381506 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:08.381648 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.381968 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.382070 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:38:08.382465 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.372085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33461 (local address 127.11.114.194:34987)
0807 06:38:08.372254 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:08.372296 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:08.373205 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:08.373415 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:08.373418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:08.373764 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:08.374033 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.374038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.374686 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.374687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.375516 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.375518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.375852 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.375852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.375953 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.376577 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:08.376592 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:08.378608 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:08.380639 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:08.380646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:08.380649 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:08.380898 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:08.381152 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:08.381153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:08.381153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:08.381275 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:08.381641 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:08.381648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:08.381817 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 06:38:08.381944 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 06:38:08.382124 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260807 06:38:08.383011 12977 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33461
I20260807 06:38:08.383119 12977 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:08.383330 12977 heartbeater.cc:511] Master 127.11.114.254:33461 requested a full tablet report, sending...
I20260807 06:38:08.383756 12532 ts_manager.cc:194] Registered new tserver with Master: 3f940789f3634797a8cd4b326ce09244 (127.11.114.194:39467)
I20260807 06:38:08.384171 12532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:34987
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:08.402649 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33461
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:08.487368 12983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:08.487637 12983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:08.487680 12983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:08.488850 12983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:08.488894 12983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:08.488922 12983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:08.490515 12983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33461
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:08.490768 12983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:08.491007 12983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:08.496237 12990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:08.496246 12991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:08.496234 12993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:08.496644 12983 server_base.cc:1034] running on GCE node
I20260807 06:38:08.496850 12983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:08.497053 12983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:08.498240 12983 hybrid_clock.cc:648] HybridClock initialized: now 1786084688498225 us; error 30 us; skew 500 ppm
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:08.500684 12983 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:08.501121 12983 webserver.cc:460] Webserver started at http://127.11.114.195:37527/ using document root <none> and password file <none>
I20260807 06:38:08.501384 12983 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:08.501436 12983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:08.501590 12983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:08.502686 12983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "c0d484d25d8c43eda573c8166357b70a"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "92085d86432155a0e0e0664b92878046"
server_key_iv: "1b6d1223b209bfdfdfdcc387c42f9dc4"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.503000 12983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "c0d484d25d8c43eda573c8166357b70a"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "92085d86432155a0e0e0664b92878046"
server_key_iv: "1b6d1223b209bfdfdfdcc387c42f9dc4"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.504273 12983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:38:08.504956 13005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.505141 12983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:08.505376 12983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "c0d484d25d8c43eda573c8166357b70a"
format_stamp: "Formatted at 2026-08-07 06:38:08 on dist-test-slave-gkw7"
server_key: "92085d86432155a0e0e0664b92878046"
server_key_iv: "1b6d1223b209bfdfdfdcc387c42f9dc4"
server_key_version: "encryptionkey@0"
I20260807 06:38:08.505489 12983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:08.517794 12983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:08.519665 12983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:08.519824 12983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:08.520059 12983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:08.520380 12983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:08.520433 12983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.520476 12983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:08.520500 12983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:08.526604 12983 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:43811
I20260807 06:38:08.526650 13161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:43811 every 8 connection(s)
I20260807 06:38:08.526997 12983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:38:08.528251 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 12983
I20260807 06:38:08.528343 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:08.528553 11723 external_mini_cluster.cc:1428] Setting key b82277ac690b7f8acaca4c61b8adaa6c
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:08.536839 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.527810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:45035 (local address 127.11.114.254:33461)
0807 06:38:08.527873 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.527875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.528091 (+ 216us) server_negotiation.cc:407] Connection header received
0807 06:38:08.529283 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.529284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.529297 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.529337 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:08.530098 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.530619 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.531715 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.531918 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.531979 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.534053 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:08.534054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:08.534054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:08.534060 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:08.535635 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.536015 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.536016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.536017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.536078 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.536304 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.536304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.536305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.536409 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:08.536459 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.536705 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.536774 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:38:08.537006 13164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.527787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33461 (local address 127.11.114.195:45035)
0807 06:38:08.527941 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:08.527984 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:08.529191 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:08.529417 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:08.529421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:08.529805 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:08.530002 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.530006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.530718 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.530719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.531617 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.531620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.531999 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.531999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.532101 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.532556 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:08.532564 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:08.533973 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:08.535717 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:08.535723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:08.535724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:08.535948 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:08.536140 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:08.536141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:08.536142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:08.536246 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:08.536463 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:08.536468 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:08.536581 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 06:38:08.536677 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 06:38:08.536798 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260807 06:38:08.537506 13162 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33461
I20260807 06:38:08.537648 13162 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:08.537832 13162 heartbeater.cc:511] Master 127.11.114.254:33461 requested a full tablet report, sending...
I20260807 06:38:08.538218 12532 ts_manager.cc:194] Registered new tserver with Master: c0d484d25d8c43eda573c8166357b70a (127.11.114.195:43811)
I20260807 06:38:08.538599 12532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:45035
I20260807 06:38:08.540584 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:38:08.548561 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.542590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54320 (local address 127.11.114.254:33461)
0807 06:38:08.542651 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.542653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.542795 (+ 142us) server_negotiation.cc:407] Connection header received
0807 06:38:08.542858 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.542859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.542870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.542925 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:08.543593 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.544130 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.544841 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.545044 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.545131 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.546257 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:08.546258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:08.546259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:08.546264 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:08.547551 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.547874 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.547874 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.547875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.547929 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.548118 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.548118 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.548119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.548265 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:08.548344 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.548430 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.548491 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 06:38:08.551537 12532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:08.553290 12532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:08.561198 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.556843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.11.114.193:42255)
0807 06:38:08.557436 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.557440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.557447 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:38:08.557504 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.557506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.557522 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.557743 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:08.557818 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.558530 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.559917 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.560697 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.560797 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.560864 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.561040 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.561103 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":110,"threads_started":1}
I20260807 06:38:08.561203 13179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.556690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42255 (local address 127.0.0.1:53280)
0807 06:38:08.557141 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:08.557152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:08.557201 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:08.557639 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:08.557639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:08.557641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:08.557670 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.557671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.558633 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.558634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.559842 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.559845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.560797 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.560798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.560889 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.560904 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:38:08.561018 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 06:38:08.561084 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260807 06:38:08.561380 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.556939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.11.114.194:39467)
0807 06:38:08.557101 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.557106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.557121 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:38:08.557169 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.557171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.557184 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.557300 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:08.557644 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.558919 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.560134 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.560753 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.560825 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.560873 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.561253 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.561302 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260807 06:38:08.561587 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.556915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:43811 (local address 127.0.0.1:33572)
0807 06:38:08.557281 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:08.557289 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:08.557322 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:08.557596 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:08.557598 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:08.557600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:08.557665 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.557667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.559217 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.559218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.560347 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.560350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.561307 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.561308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.561369 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.561373 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:08.561425 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:38:08.561470 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":75,"threads_started":1}
I20260807 06:38:08.561209 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.556690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:39467 (local address 127.0.0.1:59760)
0807 06:38:08.556881 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:08.556921 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:08.556985 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:08.557417 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:08.557421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:08.557424 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:08.557506 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.557509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.559035 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.559036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.560041 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.560043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.560910 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.560911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.560988 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.560992 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:08.561044 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:38:08.561091 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":78,"threads_started":1}
I20260807 06:38:08.561920 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.557120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.11.114.195:43811)
0807 06:38:08.557332 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.557337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.557347 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:08.557389 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.557392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.557409 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.557501 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:08.557928 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.559104 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.560440 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.561148 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.561211 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.561273 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.561762 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.561829 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":100,"threads_started":1}
I20260807 06:38:08.562397 12699 tablet_service.cc:1467] Processing CreateTablet for tablet d48bd48678b64971a55224e072db6f04 (DEFAULT_TABLE table=test-table [id=eca1f78d6a204d7aa18c400f42e4cc4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:08.562563 13072 tablet_service.cc:1467] Processing CreateTablet for tablet d48bd48678b64971a55224e072db6f04 (DEFAULT_TABLE table=test-table [id=eca1f78d6a204d7aa18c400f42e4cc4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:08.562608 12891 tablet_service.cc:1467] Processing CreateTablet for tablet d48bd48678b64971a55224e072db6f04 (DEFAULT_TABLE table=test-table [id=eca1f78d6a204d7aa18c400f42e4cc4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:08.562806 12699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48bd48678b64971a55224e072db6f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:08.562914 13072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48bd48678b64971a55224e072db6f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:08.562914 12891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48bd48678b64971a55224e072db6f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:08.566324 13187 tablet_bootstrap.cc:492] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244: Bootstrap starting.
I20260807 06:38:08.566516 13185 tablet_bootstrap.cc:492] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46: Bootstrap starting.
I20260807 06:38:08.567253 13185 tablet_bootstrap.cc:654] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:08.567253 13187 tablet_bootstrap.cc:654] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:08.568037 13187 log.cc:824] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:08.568363 13186 tablet_bootstrap.cc:492] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a: Bootstrap starting.
I20260807 06:38:08.568828 13187 tablet_bootstrap.cc:492] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244: No bootstrap required, opened a new log
I20260807 06:38:08.568902 13187 ts_tablet_manager.cc:1397] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:38:08.568925 13186 tablet_bootstrap.cc:654] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:08.569577 13185 log.cc:824] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:08.569597 13186 log.cc:824] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:08.570335 13185 tablet_bootstrap.cc:492] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46: No bootstrap required, opened a new log
I20260807 06:38:08.570394 13185 ts_tablet_manager.cc:1397] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 06:38:08.570807 13187 raft_consensus.cc:348] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.570967 13187 raft_consensus.cc:374] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:08.571013 13187 raft_consensus.cc:729] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f940789f3634797a8cd4b326ce09244, State: Initialized, Role: FOLLOWER
I20260807 06:38:08.571154 13187 consensus_queue.cc:260] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.571476 13187 ts_tablet_manager.cc:1428] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:38:08.571566 12977 heartbeater.cc:503] Master 127.11.114.254:33461 was elected leader, sending a full tablet report...
I20260807 06:38:08.572544 13185 raft_consensus.cc:348] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.572829 13185 raft_consensus.cc:374] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:08.572883 13185 raft_consensus.cc:729] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40243b43802d41d4bcc82241d72fcf46, State: Initialized, Role: FOLLOWER
I20260807 06:38:08.573002 13185 consensus_queue.cc:260] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.573335 13185 ts_tablet_manager.cc:1428] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:38:08.573336 12796 heartbeater.cc:503] Master 127.11.114.254:33461 was elected leader, sending a full tablet report...
I20260807 06:38:08.573872 13186 tablet_bootstrap.cc:492] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a: No bootstrap required, opened a new log
I20260807 06:38:08.573953 13186 ts_tablet_manager.cc:1397] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 06:38:08.576022 13186 raft_consensus.cc:348] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.576175 13186 raft_consensus.cc:374] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:08.576200 13186 raft_consensus.cc:729] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0d484d25d8c43eda573c8166357b70a, State: Initialized, Role: FOLLOWER
I20260807 06:38:08.576323 13186 consensus_queue.cc:260] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.576632 13162 heartbeater.cc:503] Master 127.11.114.254:33461 was elected leader, sending a full tablet report...
I20260807 06:38:08.576642 13186 ts_tablet_manager.cc:1428] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20260807 06:38:08.621932 12978 tablet.cc:2369] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:38:08.703754 12797 tablet.cc:2369] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:08.737816 13192 raft_consensus.cc:482] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:08.737952 13192 raft_consensus.cc:504] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.738552 13192 leader_election.cc:288] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f940789f3634797a8cd4b326ce09244 (127.11.114.194:39467), c0d484d25d8c43eda573c8166357b70a (127.11.114.195:43811)
I20260807 06:38:08.741971 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.739093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:47085 (local address 127.11.114.195:43811)
0807 06:38:08.739173 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.739175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.739185 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:08.739246 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.739248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.739257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.739344 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:08.739510 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.739975 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.740997 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.741626 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.741693 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.741703 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.741864 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.741900 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:38:08.741986 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.738993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:43811 (local address 127.11.114.193:47085)
0807 06:38:08.739104 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:08.739131 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:08.739176 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:08.739349 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:08.739350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:08.739352 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:08.739437 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.739439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.740071 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.740072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.740911 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.740913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.741692 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.741693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.741785 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.741789 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:08.741859 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:38:08.741899 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":2,"thread_start_us":49,"threads_started":1}
I20260807 06:38:08.742033 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.738871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:54089 (local address 127.11.114.194:39467)
0807 06:38:08.738941 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.738945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.739050 (+ 105us) server_negotiation.cc:407] Connection header received
0807 06:38:08.739090 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.739091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.739111 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.739218 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:08.739420 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.739988 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.741008 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.741674 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.741747 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.741756 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.741927 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.741961 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:38:08.742082 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.738763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:39467 (local address 127.11.114.193:54089)
0807 06:38:08.738917 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:08.738931 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:08.738999 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:08.739226 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:08.739228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:08.739231 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:08.739327 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.739329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.740081 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.740083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.740908 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:08.740910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.741746 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.741747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:08.741844 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.741848 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:08.741909 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 06:38:08.741953 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":5,"thread_start_us":106,"threads_started":1}
I20260807 06:38:08.742468 13095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48bd48678b64971a55224e072db6f04" candidate_uuid: "40243b43802d41d4bcc82241d72fcf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0d484d25d8c43eda573c8166357b70a" is_pre_election: true
I20260807 06:38:08.742522 12917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48bd48678b64971a55224e072db6f04" candidate_uuid: "40243b43802d41d4bcc82241d72fcf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f940789f3634797a8cd4b326ce09244" is_pre_election: true
I20260807 06:38:08.742713 13095 raft_consensus.cc:2454] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40243b43802d41d4bcc82241d72fcf46 in term 0.
I20260807 06:38:08.742713 12917 raft_consensus.cc:2454] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40243b43802d41d4bcc82241d72fcf46 in term 0.
I20260807 06:38:08.743057 12639 leader_election.cc:302] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40243b43802d41d4bcc82241d72fcf46, c0d484d25d8c43eda573c8166357b70a; no voters:
I20260807 06:38:08.743197 13192 raft_consensus.cc:2781] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:08.743291 13192 raft_consensus.cc:482] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:08.743324 13192 raft_consensus.cc:3037] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:08.744143 13192 raft_consensus.cc:504] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.744292 13192 leader_election.cc:288] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [CANDIDATE]: Term 1 election: Requested vote from peers 3f940789f3634797a8cd4b326ce09244 (127.11.114.194:39467), c0d484d25d8c43eda573c8166357b70a (127.11.114.195:43811)
I20260807 06:38:08.744515 12917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48bd48678b64971a55224e072db6f04" candidate_uuid: "40243b43802d41d4bcc82241d72fcf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f940789f3634797a8cd4b326ce09244"
I20260807 06:38:08.744535 13095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48bd48678b64971a55224e072db6f04" candidate_uuid: "40243b43802d41d4bcc82241d72fcf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0d484d25d8c43eda573c8166357b70a"
I20260807 06:38:08.744614 13095 raft_consensus.cc:3037] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:08.744620 12917 raft_consensus.cc:3037] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:08.745349 13095 raft_consensus.cc:2454] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40243b43802d41d4bcc82241d72fcf46 in term 1.
I20260807 06:38:08.745481 12917 raft_consensus.cc:2454] T d48bd48678b64971a55224e072db6f04 P 3f940789f3634797a8cd4b326ce09244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40243b43802d41d4bcc82241d72fcf46 in term 1.
I20260807 06:38:08.745508 12639 leader_election.cc:302] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40243b43802d41d4bcc82241d72fcf46, c0d484d25d8c43eda573c8166357b70a; no voters:
I20260807 06:38:08.745630 13192 raft_consensus.cc:2781] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:08.745793 13192 raft_consensus.cc:686] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [term 1 LEADER]: Becoming Leader. State: Replica: 40243b43802d41d4bcc82241d72fcf46, State: Running, Role: LEADER
I20260807 06:38:08.745908 13192 consensus_queue.cc:237] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } }
I20260807 06:38:08.746866 12532 catalog_manager.cc:5495] T d48bd48678b64971a55224e072db6f04 P 40243b43802d41d4bcc82241d72fcf46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40243b43802d41d4bcc82241d72fcf46 (127.11.114.193). New cstate: current_term: 1 leader_uuid: "40243b43802d41d4bcc82241d72fcf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f940789f3634797a8cd4b326ce09244" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40243b43802d41d4bcc82241d72fcf46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0d484d25d8c43eda573c8166357b70a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43811 } health_report { overall_health: UNKNOWN } } }
W20260807 06:38:08.777679 13163 tablet.cc:2369] T d48bd48678b64971a55224e072db6f04 P c0d484d25d8c43eda573c8166357b70a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:08.794906 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.787422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53286 (local address 127.11.114.193:42255)
0807 06:38:08.787503 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.787505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.787553 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:38:08.787598 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.787599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.787608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.787656 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:08.788175 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.788643 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.789320 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.789502 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.789625 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.792130 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:08.792143 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:08.792143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:08.792150 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:08.793870 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.794174 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.794177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.794177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.794233 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.794391 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.794391 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.794392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.794522 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:08.794645 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.794693 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.794765 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:08.819123 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.811762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.11.114.193:42255)
0807 06:38:08.811823 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.811825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.811916 (+ 91us) server_negotiation.cc:407] Connection header received
0807 06:38:08.811963 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.811964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.811974 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.812029 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:08.812559 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.813075 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.813863 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.814053 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.814108 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.816442 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:08.816443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:08.816443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:08.816449 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:08.818084 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.818475 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.818476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.818477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.818529 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.818654 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.818654 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.818654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.818733 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:08.818803 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.818939 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.819001 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[12467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:08.841221 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:08.834905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53302 (local address 127.11.114.193:42255)
0807 06:38:08.834976 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:08.834978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:08.835002 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:38:08.835073 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:08.835074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:08.835084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:08.835131 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:08.835656 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.836122 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.836802 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:08.836976 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:08.837024 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:08.839087 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:08.839089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:08.839089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:08.839096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:08.840279 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.840582 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.840583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.840583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.840633 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:08.840808 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:08.840808 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:08.840809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:08.840901 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:08.840990 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:08.841057 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:38:08.841115 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 06:38:08.842087 12671 tablet_service.cc:2908] Rejecting scan request for tablet d48bd48678b64971a55224e072db6f04: Uninitialized: clean time has not yet been initialized
I20260807 06:38:08.843236 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 12612
I20260807 06:38:08.860260 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 12802
I20260807 06:38:08.867892 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 12983
I20260807 06:38:08.876295 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 12485
2026-08-07T06:38:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2345 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[13218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[13218](info): set up 2 sockets
Aug 07 06:38:08 dist-test-slave-gkw7 krb5kdc[13218](info): commencing operation
krb5kdc: starting...
W20260807 06:38:10.133003 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:10.254026 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33169
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:33169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:10.337203 13235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:10.337411 13235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:10.337433 13235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:10.338594 13235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:10.338663 13235 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:38:10.338680 13235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:10.338694 13235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:10.338707 13235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:10.338719 13235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:10.340174 13235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:33169
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:10.340382 13235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:10.340565 13235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:10.345525 13245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:10.345618 13243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:10.345600 13235 server_base.cc:1034] running on GCE node
W20260807 06:38:10.345597 13240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:10.345953 13235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:10.346287 13235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:10.347462 13235 hybrid_clock.cc:648] HybridClock initialized: now 1786084690347436 us; error 39 us; skew 500 ppm
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:10.350090 13235 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:10.350512 13235 webserver.cc:460] Webserver started at http://127.11.114.254:44215/ using document root <none> and password file <none>
I20260807 06:38:10.350766 13235 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:10.350836 13235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:10.350986 13235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:10.352046 13235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "5a385e686b5d4d11a1fef84a922c1857"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "069dda30f177012c70c992de1907b19c"
server_key_iv: "346a879542ab0cbcc3e7f3608efeeb5c"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.352411 13235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "5a385e686b5d4d11a1fef84a922c1857"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "069dda30f177012c70c992de1907b19c"
server_key_iv: "346a879542ab0cbcc3e7f3608efeeb5c"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.353825 13235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:38:10.354539 13253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:10.354708 13235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:10.354822 13235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "5a385e686b5d4d11a1fef84a922c1857"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "069dda30f177012c70c992de1907b19c"
server_key_iv: "346a879542ab0cbcc3e7f3608efeeb5c"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.354900 13235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:10.366214 13235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:10.368101 13235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:10.368263 13235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:10.373190 13235 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:33169
I20260807 06:38:10.373298 13340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:33169 every 8 connection(s)
I20260807 06:38:10.373656 13235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:10.374383 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:10.376593 13342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857: Bootstrap starting.
I20260807 06:38:10.377144 13342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:10.377378 13342 log.cc:824] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:10.378046 13342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857: No bootstrap required, opened a new log
I20260807 06:38:10.379287 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 13235
I20260807 06:38:10.379392 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:10.379550 11723 external_mini_cluster.cc:1428] Setting key 2cb7f01adb5d2b065ae3b8f4332d9bb6
I20260807 06:38:10.380997 13342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a385e686b5d4d11a1fef84a922c1857" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33169 } }
I20260807 06:38:10.381170 13342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:10.381207 13342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a385e686b5d4d11a1fef84a922c1857, State: Initialized, Role: FOLLOWER
I20260807 06:38:10.381369 13342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a385e686b5d4d11a1fef84a922c1857" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33169 } }
I20260807 06:38:10.381451 13342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:10.381474 13342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:10.381558 13342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:10.382229 13342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a385e686b5d4d11a1fef84a922c1857" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33169 } }
I20260807 06:38:10.382336 13342 leader_election.cc:302] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a385e686b5d4d11a1fef84a922c1857; no voters:
I20260807 06:38:10.382501 13342 leader_election.cc:288] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:10.382575 13347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:10.382750 13342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:10.382750 13347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [term 1 LEADER]: Becoming Leader. State: Replica: 5a385e686b5d4d11a1fef84a922c1857, State: Running, Role: LEADER
I20260807 06:38:10.382980 13347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a385e686b5d4d11a1fef84a922c1857" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33169 } }
I20260807 06:38:10.383376 13349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a385e686b5d4d11a1fef84a922c1857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a385e686b5d4d11a1fef84a922c1857" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33169 } } }
I20260807 06:38:10.383530 13349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:10.383740 13350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a385e686b5d4d11a1fef84a922c1857. Latest consensus state: current_term: 1 leader_uuid: "5a385e686b5d4d11a1fef84a922c1857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a385e686b5d4d11a1fef84a922c1857" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33169 } } }
I20260807 06:38:10.383980 13350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:10.384151 13357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:10.384851 13357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:10.386543 13357 catalog_manager.cc:1347] Generated new cluster ID: b94adcefb3004ea89bb826e34e74a8b7
I20260807 06:38:10.386595 13357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:10.391794 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.380100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.11.114.254:33169)
0807 06:38:10.380274 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.380286 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.380324 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:38:10.380953 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.380963 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.381030 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.381206 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:10.381987 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.383417 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.384575 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.385141 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.385264 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.387998 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:10.388002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:10.388002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:10.388015 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:10.390045 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.390455 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.390459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.390459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.390654 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.390870 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.390872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.390872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.391192 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:10.391367 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.391438 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.391577 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260807 06:38:10.402829 13357 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:10.403395 13357 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:10.409281 13357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857: Generated new TSK 0
I20260807 06:38:10.409475 13357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:10.444129 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33169
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:10.551311 13370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:10.551543 13370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:10.551590 13370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:10.552801 13370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:10.552868 13370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:10.552901 13370 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:10.552943 13370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:10.554662 13370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:33169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:10.554953 13370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:10.555233 13370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:10.560436 13380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:10.560525 13382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:10.560580 13377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:10.560752 13370 server_base.cc:1034] running on GCE node
I20260807 06:38:10.560997 13370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:10.561242 13370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:10.562414 13370 hybrid_clock.cc:648] HybridClock initialized: now 1786084690562406 us; error 45 us; skew 500 ppm
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:10.565102 13370 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:10.565577 13370 webserver.cc:460] Webserver started at http://127.11.114.193:42075/ using document root <none> and password file <none>
I20260807 06:38:10.565833 13370 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:10.565902 13370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:10.566049 13370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:10.567126 13370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "81cefe2c45874ed6a5fae197644e8164"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "bf2da1f2dee4dd9e4b0a61b28d40c0e2"
server_key_iv: "176a48f2edbc8338c88c3c427df8b4b3"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.567490 13370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "81cefe2c45874ed6a5fae197644e8164"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "bf2da1f2dee4dd9e4b0a61b28d40c0e2"
server_key_iv: "176a48f2edbc8338c88c3c427df8b4b3"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.568852 13370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:10.569684 13394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:10.569892 13370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:10.569994 13370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "81cefe2c45874ed6a5fae197644e8164"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "bf2da1f2dee4dd9e4b0a61b28d40c0e2"
server_key_iv: "176a48f2edbc8338c88c3c427df8b4b3"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.570078 13370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:10.582827 13370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:10.585283 13370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:10.585448 13370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:10.585997 13370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:10.586067 13370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:10.586110 13370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:10.586143 13370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:10.593323 13370 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:33299
I20260807 06:38:10.593407 13556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:33299 every 8 connection(s)
I20260807 06:38:10.593770 13370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:10.597160 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.586655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:44645 (local address 127.11.114.254:33169)
0807 06:38:10.586718 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.586720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.586954 (+ 234us) server_negotiation.cc:407] Connection header received
0807 06:38:10.587627 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.587629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.587649 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.587720 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:10.588660 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.589509 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.591176 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.591538 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.591627 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.594448 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:10.594449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:10.594450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:10.594460 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:10.595930 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.596337 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.596339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.596339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.596391 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.596620 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.596621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.596621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.596723 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:10.596775 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.597024 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.597097 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:38:10.597419 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.586642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33169 (local address 127.11.114.193:44645)
0807 06:38:10.586826 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.586862 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.587467 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.587796 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.587800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.588222 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:10.588527 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.588535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.589627 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.589628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.591064 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.591068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.591634 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.591636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.591744 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.592387 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:10.592399 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:10.594346 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:10.596015 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.596030 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.596032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.596268 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.596451 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.596452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.596452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.596561 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.596791 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:10.596796 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:10.596909 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.596998 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.597148 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260807 06:38:10.597960 13557 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33169
I20260807 06:38:10.598078 13557 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:10.598268 13557 heartbeater.cc:511] Master 127.11.114.254:33169 requested a full tablet report, sending...
I20260807 06:38:10.598929 13277 ts_manager.cc:194] Registered new tserver with Master: 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299)
I20260807 06:38:10.599646 13277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:44645
I20260807 06:38:10.601629 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 13370
I20260807 06:38:10.601737 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:10.601929 11723 external_mini_cluster.cc:1428] Setting key 95078bd8f4cef7b461204b98a76aeac8
I20260807 06:38:10.606643 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.599227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33169 (local address 127.11.114.193:59301)
0807 06:38:10.599379 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.599389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.599433 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.600229 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.600230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.600450 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:10.600549 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.600551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.601221 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.601222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.602009 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.602012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.602528 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.602529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.602622 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.603080 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:10.603085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:10.603401 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:10.605383 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.605384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.605384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.605671 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.605929 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.605931 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.605931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.606006 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.606317 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:10.606318 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:10.606376 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.606443 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.606539 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260807 06:38:10.606739 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.599852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:59301 (local address 127.11.114.254:33169)
0807 06:38:10.600095 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.600097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.600107 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:10.600129 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.600131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.600143 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.600211 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:10.600654 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.601109 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.602105 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.602406 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.602493 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.603516 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:10.603517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:10.603518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:10.603533 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:10.605282 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.605768 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.605769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.605770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.605842 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.606086 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.606087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.606088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.606238 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:10.606320 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.606493 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.606599 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:10.630715 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33169
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:10.714277 13568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:10.714481 13568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:10.714505 13568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:10.715734 13568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:10.715775 13568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:10.715795 13568 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:10.715813 13568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:10.717319 13568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:33169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:10.717609 13568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:10.717859 13568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:10.722997 13574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:10.722983 13575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:10.723140 13579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:10.723333 13568 server_base.cc:1034] running on GCE node
I20260807 06:38:10.723536 13568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:10.723795 13568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:10.724959 13568 hybrid_clock.cc:648] HybridClock initialized: now 1786084690724943 us; error 28 us; skew 500 ppm
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:10.727512 13568 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:10.727949 13568 webserver.cc:460] Webserver started at http://127.11.114.194:34807/ using document root <none> and password file <none>
I20260807 06:38:10.728235 13568 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:10.728305 13568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:10.728456 13568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:10.729583 13568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "ba8aeacd350241158ad0ec7fc008a3d7"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "c2159fbc39fcd61eb436f57694daf7cf"
server_key_iv: "b5063fe366849a041f4a3b011c164ca5"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.729961 13568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "ba8aeacd350241158ad0ec7fc008a3d7"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "c2159fbc39fcd61eb436f57694daf7cf"
server_key_iv: "b5063fe366849a041f4a3b011c164ca5"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.731346 13568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:10.732148 13591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:10.732354 13568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:10.732455 13568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "ba8aeacd350241158ad0ec7fc008a3d7"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "c2159fbc39fcd61eb436f57694daf7cf"
server_key_iv: "b5063fe366849a041f4a3b011c164ca5"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.732542 13568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:10.777410 13568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:10.779469 13568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:10.779640 13568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:10.780191 13568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:10.780265 13568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:10.780334 13568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:10.780364 13568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:10.787845 13568 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:39169
I20260807 06:38:10.787930 13759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:39169 every 8 connection(s)
I20260807 06:38:10.788264 13568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:10.790804 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.780801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:55571 (local address 127.11.114.254:33169)
0807 06:38:10.780885 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.780888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.781085 (+ 197us) server_negotiation.cc:407] Connection header received
0807 06:38:10.781654 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.781656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.781672 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.781784 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:10.782661 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.783596 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.784777 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.785117 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.785202 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.788086 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:10.788088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:10.788088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:10.788096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:10.789628 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.790005 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.790006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.790006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.790059 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.790288 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.790289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.790289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.790391 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:10.790442 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.790671 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.790742 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:38:10.791029 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.780762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33169 (local address 127.11.114.194:55571)
0807 06:38:10.780953 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.780994 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.781499 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.781853 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.781856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.782245 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:10.782549 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.782557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.783717 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.783718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.784688 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.784691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.785229 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.785230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.785409 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.786001 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:10.786010 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:10.787996 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:10.789698 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.789714 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.789716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.789938 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.790118 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.790119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.790119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.790232 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.790446 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:10.790451 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:10.790560 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.790643 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.790778 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
I20260807 06:38:10.791559 13761 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33169
I20260807 06:38:10.791638 13761 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:10.791846 13761 heartbeater.cc:511] Master 127.11.114.254:33169 requested a full tablet report, sending...
I20260807 06:38:10.792266 13278 ts_manager.cc:194] Registered new tserver with Master: ba8aeacd350241158ad0ec7fc008a3d7 (127.11.114.194:39169)
I20260807 06:38:10.792701 13278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:55571
I20260807 06:38:10.796798 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 13568
I20260807 06:38:10.796898 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:10.797097 11723 external_mini_cluster.cc:1428] Setting key e83fb59613d6fc349e1cdf5cbef0dde5
I20260807 06:38:10.798377 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.792666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33169 (local address 127.11.114.194:57629)
0807 06:38:10.792873 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.792885 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.792938 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.793053 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.793054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.793201 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:10.793273 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.793274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.793955 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.793956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.794479 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.794480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.794776 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.794777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.794830 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.795191 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:10.795195 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:10.795464 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:10.797179 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.797179 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.797180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.797407 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.797628 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.797629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.797629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.797700 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.798035 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:10.798035 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:10.798074 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.798127 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.798252 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
I20260807 06:38:10.798396 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.792746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:57629 (local address 127.11.114.254:33169)
0807 06:38:10.792792 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.792794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.792924 (+ 130us) server_negotiation.cc:407] Connection header received
0807 06:38:10.792993 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.792993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.793004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.793052 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:10.793351 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.793877 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.794543 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.794717 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.794766 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.795538 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:10.795539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:10.795539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:10.795545 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:10.797095 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.797474 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.797475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.797475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.797575 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.797771 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.797772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.797772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.797963 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:10.798042 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.798174 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.798257 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:10.822839 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33169
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:10.907195 13770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:10.907430 13770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:10.907470 13770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:10.908622 13770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:10.908684 13770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:10.908716 13770 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:10.908759 13770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:10.910210 13770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:33169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:10.910475 13770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:10.910697 13770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:10.915841 13777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:10.915922 13770 server_base.cc:1034] running on GCE node
W20260807 06:38:10.915886 13778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:10.915841 13782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:10.916175 13770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:10.916420 13770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:10.917627 13770 hybrid_clock.cc:648] HybridClock initialized: now 1786084690917616 us; error 37 us; skew 500 ppm
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:10.924137 13770 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:10.924613 13770 webserver.cc:460] Webserver started at http://127.11.114.195:45949/ using document root <none> and password file <none>
I20260807 06:38:10.924857 13770 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:10.924930 13770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:10.925097 13770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:10.926293 13770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "1ef76a28697f4e488c034faf4c14c2c1"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "f47b47654aee5f49664656bf9ae7ac3c"
server_key_iv: "a292853cd4df1564f6670722be8ad635"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.926697 13770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "1ef76a28697f4e488c034faf4c14c2c1"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "f47b47654aee5f49664656bf9ae7ac3c"
server_key_iv: "a292853cd4df1564f6670722be8ad635"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.928216 13770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:10.929087 13792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:10.929261 13770 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:10.929380 13770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "1ef76a28697f4e488c034faf4c14c2c1"
format_stamp: "Formatted at 2026-08-07 06:38:10 on dist-test-slave-gkw7"
server_key: "f47b47654aee5f49664656bf9ae7ac3c"
server_key_iv: "a292853cd4df1564f6670722be8ad635"
server_key_version: "encryptionkey@0"
I20260807 06:38:10.929474 13770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:10.936784 13770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:10.939384 13770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:10.939565 13770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:10.940111 13770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:10.940179 13770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:10.940225 13770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:10.940253 13770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:10.948443 13770 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:34179
I20260807 06:38:10.948880 13954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:34179 every 8 connection(s)
I20260807 06:38:10.948931 13770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:38:10.951041 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.940581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:49861 (local address 127.11.114.254:33169)
0807 06:38:10.940661 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.940664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.940839 (+ 175us) server_negotiation.cc:407] Connection header received
0807 06:38:10.941300 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.941302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.941319 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.941413 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:10.942085 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.943071 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.944732 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.945143 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.945239 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.947903 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:10.947905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:10.947905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:10.947914 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:10.949662 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.950061 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.950062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.950062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.950125 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.950400 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.950401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.950401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.950546 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:10.950614 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.950911 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.950978 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:38:10.951241 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.940547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33169 (local address 127.11.114.195:49861)
0807 06:38:10.940696 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.940747 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.941200 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.941477 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.941481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.941798 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:10.941980 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.941985 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.943187 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.943188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.944630 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.944633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.945260 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.945262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.945407 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.945992 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:10.946001 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:10.947804 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:10.949757 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.949763 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.949765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.949981 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.950196 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.950197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.950198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.950310 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.950640 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:10.950654 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:10.950778 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.950883 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.951014 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260807 06:38:10.951830 13956 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33169
I20260807 06:38:10.951928 13956 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:10.952186 13956 heartbeater.cc:511] Master 127.11.114.254:33169 requested a full tablet report, sending...
I20260807 06:38:10.952661 13277 ts_manager.cc:194] Registered new tserver with Master: 1ef76a28697f4e488c034faf4c14c2c1 (127.11.114.195:34179)
I20260807 06:38:10.953075 13277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:49861
I20260807 06:38:10.958658 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 13770
I20260807 06:38:10.958691 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.953010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33169 (local address 127.11.114.195:45065)
0807 06:38:10.953190 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.953201 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.953265 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.953435 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.953436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.953714 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:10.953792 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.953795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.954455 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.954455 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.954964 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.954965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.955298 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.955299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.955370 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.955772 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:10.955776 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:10.956053 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:10.957745 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.957746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.957746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.957911 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.958085 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:10.958086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:10.958086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:10.958144 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:10.958423 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:10.958424 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:10.958470 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.958533 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.958620 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
I20260807 06:38:10.958756 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.953074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:45065 (local address 127.11.114.254:33169)
0807 06:38:10.953123 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.953125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.953268 (+ 143us) server_negotiation.cc:407] Connection header received
0807 06:38:10.953330 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.953331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.953343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.953418 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:10.953877 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.954357 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.955037 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.955210 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.955266 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.956145 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:10.956147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:10.956147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:10.956153 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:10.957674 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.957975 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.957976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.957976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.958025 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.958195 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.958196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.958196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.958334 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:10.958396 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.958580 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.958658 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:10.958763 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:10.958933 11723 external_mini_cluster.cc:1428] Setting key de516d4f60c475634c6c7c95b0cd8616
I20260807 06:38:10.959885 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:38:10 dist-test-slave-gkw7 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:10.984237 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.976794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.11.114.254:33169)
0807 06:38:10.976865 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.976868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.976878 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:10.976933 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.976934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.976949 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.977015 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:10.977637 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.978139 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.978900 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.979093 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.979169 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.981625 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:10.981627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:10.981627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:10.981634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:10.982964 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.983315 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.983319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.983319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.983375 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:10.983624 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:10.983625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:10.983625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:10.983743 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:10.983857 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.984041 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.984120 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:38:10.986032 13278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:10.987531 13278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:10.994334 13974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.990198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:33299 (local address 127.0.0.1:42710)
0807 06:38:10.990357 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.990367 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.990406 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.990816 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.990817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.990819 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:10.990866 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.990867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.992086 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.992087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.993265 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.993267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.994025 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.994026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.994103 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.994116 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.994189 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.994245 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":43,"thread_start_us":95,"threads_started":1}
I20260807 06:38:10.994349 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.990442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.11.114.193:33299)
0807 06:38:10.990643 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.990647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.990657 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:10.990696 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.990698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.990713 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.990935 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:10.990994 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.992040 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.993349 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.993925 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.994025 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.994086 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.994213 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.994257 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260807 06:38:10.994567 13982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.989946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:39169 (local address 127.0.0.1:55748)
0807 06:38:10.990335 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.990346 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.990403 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.990738 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.990739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.990741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:10.990792 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.990793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.991840 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.991841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.993316 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.993319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.994333 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.994334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.994393 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.994398 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.994433 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.994487 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":67,"thread_start_us":78,"threads_started":1}
I20260807 06:38:10.995015 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.990055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.11.114.194:39169)
0807 06:38:10.990533 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.990538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.990550 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:38:10.990607 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.990609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.990627 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.990896 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:10.990956 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.991751 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.993447 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.994243 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.994792 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.994852 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.994876 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.994914 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":88,"threads_started":1}
I20260807 06:38:10.995428 13465 tablet_service.cc:1467] Processing CreateTablet for tablet 522493be416d471a9a9a4bd43f2a1938 (DEFAULT_TABLE table=test-table [id=0dd2e4394952454c917a2187786a601e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:10.995666 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.989944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34179 (local address 127.0.0.1:60532)
0807 06:38:10.990122 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:10.990162 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:10.990228 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:10.990517 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:10.990526 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:10.990528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:10.990617 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.990619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.991916 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.991916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.994047 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:10.994050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.995422 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.995423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:10.995500 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.995504 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:10.995560 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:38:10.995599 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":1010,"thread_start_us":64,"threads_started":1}
I20260807 06:38:10.995718 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:10.990129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.11.114.195:34179)
0807 06:38:10.990281 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:10.990286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:10.990298 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:38:10.990343 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:10.990346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:10.990365 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:10.990443 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:10.990713 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.991851 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.994154 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:10.995318 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:10.995421 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:10.995473 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:10.995575 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 06:38:10.995620 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260807 06:38:10.995643 13669 tablet_service.cc:1467] Processing CreateTablet for tablet 522493be416d471a9a9a4bd43f2a1938 (DEFAULT_TABLE table=test-table [id=0dd2e4394952454c917a2187786a601e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:10.995836 13465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 522493be416d471a9a9a4bd43f2a1938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:10.995955 13669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 522493be416d471a9a9a4bd43f2a1938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:10.996259 13865 tablet_service.cc:1467] Processing CreateTablet for tablet 522493be416d471a9a9a4bd43f2a1938 (DEFAULT_TABLE table=test-table [id=0dd2e4394952454c917a2187786a601e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:10.996527 13865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 522493be416d471a9a9a4bd43f2a1938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:10.999279 13986 tablet_bootstrap.cc:492] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Bootstrap starting.
I20260807 06:38:10.999583 13987 tablet_bootstrap.cc:492] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Bootstrap starting.
I20260807 06:38:11.000316 13986 tablet_bootstrap.cc:654] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.000316 13987 tablet_bootstrap.cc:654] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.000676 13986 log.cc:824] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:11.000676 13987 log.cc:824] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:11.001348 13986 tablet_bootstrap.cc:492] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: No bootstrap required, opened a new log
I20260807 06:38:11.001384 13987 tablet_bootstrap.cc:492] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: No bootstrap required, opened a new log
I20260807 06:38:11.001394 13986 ts_tablet_manager.cc:1397] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:38:11.001432 13987 ts_tablet_manager.cc:1397] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:11.002847 13990 tablet_bootstrap.cc:492] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Bootstrap starting.
I20260807 06:38:11.003217 13986 raft_consensus.cc:348] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.003388 13986 raft_consensus.cc:374] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.003433 13986 raft_consensus.cc:729] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76a28697f4e488c034faf4c14c2c1, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.003538 13986 consensus_queue.cc:260] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.003618 13990 tablet_bootstrap.cc:654] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.003628 13987 raft_consensus.cc:348] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.003815 13987 raft_consensus.cc:374] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.003834 13986 ts_tablet_manager.cc:1428] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:38:11.003873 13987 raft_consensus.cc:729] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cefe2c45874ed6a5fae197644e8164, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.003994 13990 log.cc:824] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:11.003997 13987 consensus_queue.cc:260] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.004261 13987 ts_tablet_manager.cc:1428] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:38:11.004308 13557 heartbeater.cc:503] Master 127.11.114.254:33169 was elected leader, sending a full tablet report...
I20260807 06:38:11.004382 13956 heartbeater.cc:503] Master 127.11.114.254:33169 was elected leader, sending a full tablet report...
I20260807 06:38:11.004940 13990 tablet_bootstrap.cc:492] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: No bootstrap required, opened a new log
I20260807 06:38:11.005097 13990 ts_tablet_manager.cc:1397] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:11.006875 13990 raft_consensus.cc:348] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.007030 13990 raft_consensus.cc:374] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.007095 13990 raft_consensus.cc:729] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8aeacd350241158ad0ec7fc008a3d7, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.007223 13990 consensus_queue.cc:260] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.007486 13990 ts_tablet_manager.cc:1428] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:38:11.007573 13761 heartbeater.cc:503] Master 127.11.114.254:33169 was elected leader, sending a full tablet report...
W20260807 06:38:11.038893 13762 tablet.cc:2369] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:38:11.094573 13558 tablet.cc:2369] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:38:11.199609 13958 tablet.cc:2369] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:11.209985 13996 raft_consensus.cc:482] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:11.210124 13996 raft_consensus.cc:504] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.210443 13996 leader_election.cc:288] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ef76a28697f4e488c034faf4c14c2c1 (127.11.114.195:34179), 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299)
I20260807 06:38:11.214666 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.210844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:44141 (local address 127.11.114.195:34179)
0807 06:38:11.210923 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.210926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.210938 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:38:11.210974 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.210975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.210987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.211062 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:11.211259 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.212161 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.213653 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.214319 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.214391 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.214400 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.214545 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.214589 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:38:11.214654 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.210689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34179 (local address 127.11.114.194:44141)
0807 06:38:11.210785 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:11.210823 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:11.210895 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:11.211083 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:11.211084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:11.211085 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:11.211166 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.211168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.212248 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.212249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.213554 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.213558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.214391 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.214391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.214470 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.214473 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:11.214525 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:38:11.214556 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":40}
I20260807 06:38:11.214679 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.211024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:43593 (local address 127.11.114.193:33299)
0807 06:38:11.211079 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.211082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.211091 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:11.211148 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.211149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.211157 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.211197 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:11.211365 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.212147 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.213654 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.214271 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.214361 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.214373 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.214551 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.214598 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 06:38:11.214664 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.210825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:33299 (local address 127.11.114.194:43593)
0807 06:38:11.211017 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:11.211025 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:11.211086 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:11.211216 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:11.211217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:11.211218 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:11.211295 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.211296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.212251 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.212252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.213559 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.213561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.214374 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.214376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.214451 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.214456 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:11.214516 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:38:11.214556 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":5,"thread_start_us":121,"threads_started":1}
I20260807 06:38:11.215116 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "522493be416d471a9a9a4bd43f2a1938" candidate_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef76a28697f4e488c034faf4c14c2c1" is_pre_election: true
I20260807 06:38:11.215094 13495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "522493be416d471a9a9a4bd43f2a1938" candidate_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81cefe2c45874ed6a5fae197644e8164" is_pre_election: true
I20260807 06:38:11.215266 13495 raft_consensus.cc:2454] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aeacd350241158ad0ec7fc008a3d7 in term 0.
I20260807 06:38:11.215273 13890 raft_consensus.cc:2454] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aeacd350241158ad0ec7fc008a3d7 in term 0.
I20260807 06:38:11.215485 13594 leader_election.cc:302] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81cefe2c45874ed6a5fae197644e8164, ba8aeacd350241158ad0ec7fc008a3d7; no voters:
I20260807 06:38:11.215667 13996 raft_consensus.cc:2781] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:11.215739 13996 raft_consensus.cc:482] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:11.215759 13996 raft_consensus.cc:3037] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.217414 13996 raft_consensus.cc:504] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.217588 13996 leader_election.cc:288] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [CANDIDATE]: Term 1 election: Requested vote from peers 1ef76a28697f4e488c034faf4c14c2c1 (127.11.114.195:34179), 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299)
I20260807 06:38:11.217741 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "522493be416d471a9a9a4bd43f2a1938" candidate_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef76a28697f4e488c034faf4c14c2c1"
I20260807 06:38:11.217806 13495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "522493be416d471a9a9a4bd43f2a1938" candidate_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81cefe2c45874ed6a5fae197644e8164"
I20260807 06:38:11.217873 13890 raft_consensus.cc:3037] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.217869 13495 raft_consensus.cc:3037] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.218598 13890 raft_consensus.cc:2454] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aeacd350241158ad0ec7fc008a3d7 in term 1.
I20260807 06:38:11.218593 13495 raft_consensus.cc:2454] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aeacd350241158ad0ec7fc008a3d7 in term 1.
I20260807 06:38:11.218778 13594 leader_election.cc:302] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81cefe2c45874ed6a5fae197644e8164, ba8aeacd350241158ad0ec7fc008a3d7; no voters:
I20260807 06:38:11.218933 13996 raft_consensus.cc:2781] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:11.219095 13996 raft_consensus.cc:686] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 LEADER]: Becoming Leader. State: Replica: ba8aeacd350241158ad0ec7fc008a3d7, State: Running, Role: LEADER
I20260807 06:38:11.219177 13996 consensus_queue.cc:237] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } }
I20260807 06:38:11.220171 13277 catalog_manager.cc:5495] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba8aeacd350241158ad0ec7fc008a3d7 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } health_report { overall_health: UNKNOWN } } }
I20260807 06:38:11.227072 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.224421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.11.114.194:39169)
0807 06:38:11.224470 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.224472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.224482 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:11.224555 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.224555 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.224578 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.224650 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:11.224821 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.225285 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.226189 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.226386 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.226485 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.226707 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:11.226804 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:11.226887 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.226943 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.226965 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:11.228915 13890 raft_consensus.cc:1261] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Refusing update from remote peer ba8aeacd350241158ad0ec7fc008a3d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:11.229328 13999 consensus_queue.cc:1035] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:11.231086 13495 raft_consensus.cc:1261] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Refusing update from remote peer ba8aeacd350241158ad0ec7fc008a3d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:11.231405 13996 consensus_queue.cc:1035] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:11.232548 14004 mvcc.cc:205] Tried to move back new op lower bound from 7315802895270576128 to 7315802895234314240. Current Snapshot: MvccSnapshot[applied={T|T < 7315802895270576128}]
I20260807 06:38:11.237022 13278 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:11.237138 13278 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:11.238082 13278 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857: Sending DeleteTablet for 3 replicas of tablet 522493be416d471a9a9a4bd43f2a1938
I20260807 06:38:11.238590 13465 tablet_service.cc:1514] Processing DeleteTablet for tablet 522493be416d471a9a9a4bd43f2a1938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:42710
I20260807 06:38:11.238601 13865 tablet_service.cc:1514] Processing DeleteTablet for tablet 522493be416d471a9a9a4bd43f2a1938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:60532
I20260807 06:38:11.238821 14014 tablet_replica.cc:330] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: stopping tablet replica
I20260807 06:38:11.238895 13669 tablet_service.cc:1514] Processing DeleteTablet for tablet 522493be416d471a9a9a4bd43f2a1938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:55748
I20260807 06:38:11.238943 14014 raft_consensus.cc:2229] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:11.239053 14014 raft_consensus.cc:2258] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:11.239243 14016 tablet_replica.cc:330] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: stopping tablet replica
I20260807 06:38:11.239210 13278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:11.239365 13278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:11.239405 14016 raft_consensus.cc:2229] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:11.239511 14016 raft_consensus.cc:2258] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:11.239609 14014 ts_tablet_manager.cc:1905] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:11.239984 14016 ts_tablet_manager.cc:1905] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:11.241379 13865 tablet_service.cc:1467] Processing CreateTablet for tablet 79235d47b93340ba8533080d119ed3b3 (DEFAULT_TABLE table=test-table [id=c83c6be8020349b38967fa79c088d4d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:11.241513 13865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79235d47b93340ba8533080d119ed3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:11.241595 14014 ts_tablet_manager.cc:1918] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:11.241674 14014 log.cc:1196] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/522493be416d471a9a9a4bd43f2a1938
I20260807 06:38:11.241986 14014 ts_tablet_manager.cc:1939] T 522493be416d471a9a9a4bd43f2a1938 P 81cefe2c45874ed6a5fae197644e8164: Deleting consensus metadata
I20260807 06:38:11.242046 14016 ts_tablet_manager.cc:1918] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:11.242112 14016 log.cc:1196] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/522493be416d471a9a9a4bd43f2a1938
I20260807 06:38:11.242409 14016 ts_tablet_manager.cc:1939] T 522493be416d471a9a9a4bd43f2a1938 P ba8aeacd350241158ad0ec7fc008a3d7: Deleting consensus metadata
I20260807 06:38:11.242997 13986 tablet_bootstrap.cc:492] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: Bootstrap starting.
I20260807 06:38:11.243044 13259 catalog_manager.cc:4841] TS ba8aeacd350241158ad0ec7fc008a3d7 (127.11.114.194:39169): tablet 522493be416d471a9a9a4bd43f2a1938 (table test-table [id=0dd2e4394952454c917a2187786a601e]) successfully deleted
I20260807 06:38:11.243269 13256 catalog_manager.cc:4841] TS 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299): tablet 522493be416d471a9a9a4bd43f2a1938 (table test-table [id=0dd2e4394952454c917a2187786a601e]) successfully deleted
I20260807 06:38:11.243486 13986 tablet_bootstrap.cc:654] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.243681 13465 tablet_service.cc:1467] Processing CreateTablet for tablet 79235d47b93340ba8533080d119ed3b3 (DEFAULT_TABLE table=test-table [id=c83c6be8020349b38967fa79c088d4d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:11.243803 13465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79235d47b93340ba8533080d119ed3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:11.244994 13987 tablet_bootstrap.cc:492] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: Bootstrap starting.
I20260807 06:38:11.245137 13669 tablet_service.cc:1467] Processing CreateTablet for tablet 79235d47b93340ba8533080d119ed3b3 (DEFAULT_TABLE table=test-table [id=c83c6be8020349b38967fa79c088d4d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:11.245271 13669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79235d47b93340ba8533080d119ed3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:11.245396 13987 tablet_bootstrap.cc:654] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.245759 14015 tablet_replica.cc:330] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: stopping tablet replica
I20260807 06:38:11.245872 14015 raft_consensus.cc:2229] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:11.246023 14015 raft_consensus.cc:2258] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:11.246498 13990 tablet_bootstrap.cc:492] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: Bootstrap starting.
I20260807 06:38:11.246551 13986 tablet_bootstrap.cc:492] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: No bootstrap required, opened a new log
I20260807 06:38:11.246594 13986 ts_tablet_manager.cc:1397] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:38:11.246605 14015 ts_tablet_manager.cc:1905] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:11.246768 13986 raft_consensus.cc:348] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.246850 13986 raft_consensus.cc:374] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.246874 13986 raft_consensus.cc:729] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76a28697f4e488c034faf4c14c2c1, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.246904 13990 tablet_bootstrap.cc:654] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.246934 13986 consensus_queue.cc:260] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.247032 13986 ts_tablet_manager.cc:1428] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:11.248126 13990 tablet_bootstrap.cc:492] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: No bootstrap required, opened a new log
I20260807 06:38:11.248174 13990 ts_tablet_manager.cc:1397] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:11.248297 14015 ts_tablet_manager.cc:1918] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:11.248368 14015 log.cc:1196] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/522493be416d471a9a9a4bd43f2a1938
I20260807 06:38:11.248327 13990 raft_consensus.cc:348] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.248391 13990 raft_consensus.cc:374] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.248412 13990 raft_consensus.cc:729] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8aeacd350241158ad0ec7fc008a3d7, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.248453 13990 consensus_queue.cc:260] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.248528 13990 ts_tablet_manager.cc:1428] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:11.248566 14015 ts_tablet_manager.cc:1939] T 522493be416d471a9a9a4bd43f2a1938 P 1ef76a28697f4e488c034faf4c14c2c1: Deleting consensus metadata
I20260807 06:38:11.248941 13256 catalog_manager.cc:4841] TS 1ef76a28697f4e488c034faf4c14c2c1 (127.11.114.195:34179): tablet 522493be416d471a9a9a4bd43f2a1938 (table test-table [id=0dd2e4394952454c917a2187786a601e]) successfully deleted
I20260807 06:38:11.249485 13987 tablet_bootstrap.cc:492] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: No bootstrap required, opened a new log
I20260807 06:38:11.249560 13987 ts_tablet_manager.cc:1397] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 06:38:11.249706 13987 raft_consensus.cc:348] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.249778 13987 raft_consensus.cc:374] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.249800 13987 raft_consensus.cc:729] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cefe2c45874ed6a5fae197644e8164, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.249837 13987 consensus_queue.cc:260] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.249943 13987 ts_tablet_manager.cc:1428] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:11.280854 13996 raft_consensus.cc:482] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:11.280944 13996 raft_consensus.cc:504] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.281124 13996 leader_election.cc:288] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299), 1ef76a28697f4e488c034faf4c14c2c1 (127.11.114.195:34179)
I20260807 06:38:11.281369 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79235d47b93340ba8533080d119ed3b3" candidate_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef76a28697f4e488c034faf4c14c2c1" is_pre_election: true
I20260807 06:38:11.281473 13890 raft_consensus.cc:2454] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aeacd350241158ad0ec7fc008a3d7 in term 0.
I20260807 06:38:11.281426 13495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79235d47b93340ba8533080d119ed3b3" candidate_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81cefe2c45874ed6a5fae197644e8164" is_pre_election: true
I20260807 06:38:11.281548 13495 raft_consensus.cc:2454] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aeacd350241158ad0ec7fc008a3d7 in term 0.
I20260807 06:38:11.281682 13594 leader_election.cc:302] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ef76a28697f4e488c034faf4c14c2c1, ba8aeacd350241158ad0ec7fc008a3d7; no voters:
I20260807 06:38:11.281842 13996 raft_consensus.cc:2781] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:11.281900 13996 raft_consensus.cc:482] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:11.281932 13996 raft_consensus.cc:3037] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.282678 13996 raft_consensus.cc:504] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.282809 13996 leader_election.cc:288] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [CANDIDATE]: Term 1 election: Requested vote from peers 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299), 1ef76a28697f4e488c034faf4c14c2c1 (127.11.114.195:34179)
I20260807 06:38:11.282967 13890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79235d47b93340ba8533080d119ed3b3" candidate_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef76a28697f4e488c034faf4c14c2c1"
I20260807 06:38:11.282974 13495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79235d47b93340ba8533080d119ed3b3" candidate_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81cefe2c45874ed6a5fae197644e8164"
I20260807 06:38:11.283062 13495 raft_consensus.cc:3037] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.283064 13890 raft_consensus.cc:3037] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.283887 13890 raft_consensus.cc:2454] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aeacd350241158ad0ec7fc008a3d7 in term 1.
I20260807 06:38:11.284062 13594 leader_election.cc:302] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ef76a28697f4e488c034faf4c14c2c1, ba8aeacd350241158ad0ec7fc008a3d7; no voters:
I20260807 06:38:11.284157 13996 raft_consensus.cc:2781] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:11.284224 13996 raft_consensus.cc:686] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 LEADER]: Becoming Leader. State: Replica: ba8aeacd350241158ad0ec7fc008a3d7, State: Running, Role: LEADER
I20260807 06:38:11.284274 13996 consensus_queue.cc:237] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.284332 13495 raft_consensus.cc:2454] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aeacd350241158ad0ec7fc008a3d7 in term 1.
I20260807 06:38:11.284837 13278 catalog_manager.cc:5495] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba8aeacd350241158ad0ec7fc008a3d7 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } health_report { overall_health: UNKNOWN } } }
Aug 07 06:38:11 dist-test-slave-gkw7 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:11.318588 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.310461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.11.114.254:33169)
0807 06:38:11.310534 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.310536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.310670 (+ 134us) server_negotiation.cc:407] Connection header received
0807 06:38:11.310746 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.310747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.310760 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.310831 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:11.311333 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.311845 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.312565 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.312753 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.312839 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.315328 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:11.315330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:11.315330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:11.315339 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:11.317067 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:11.317558 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:11.317559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:11.317560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:11.317647 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:11.317895 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:11.317895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:11.317896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:11.318031 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:11.318127 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.318321 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.318450 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:38:11.318679 14031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.310394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33169 (local address 127.0.0.1:47900)
0807 06:38:11.310586 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:11.310601 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:11.310689 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:11.310838 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:11.310839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:11.311190 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:11.311240 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.311243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.311929 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.311930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.312480 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.312482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.312839 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.312840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.312896 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.313456 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:11.313461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:11.315219 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:11.317188 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:11.317197 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:11.317200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:11.317424 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:11.317730 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:11.317731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:11.317732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:11.317810 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:11.318115 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:11.318123 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:11.318178 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 06:38:11.318291 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 06:38:11.318523 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":109,"thread_start_us":90,"threads_started":1}
I20260807 06:38:11.321611 13278 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:47900:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:38:11.321940 13278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:11.323525 13669 tablet_service.cc:1467] Processing CreateTablet for tablet 33ffb6e7e93940e5a0c665a045fd2c7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d401e0705adf45fbac920ae0a465d466]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:11.323760 13669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ffb6e7e93940e5a0c665a045fd2c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:11.325464 13990 tablet_bootstrap.cc:492] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: Bootstrap starting.
I20260807 06:38:11.325609 13465 tablet_service.cc:1467] Processing CreateTablet for tablet 33ffb6e7e93940e5a0c665a045fd2c7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d401e0705adf45fbac920ae0a465d466]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:11.325757 13465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ffb6e7e93940e5a0c665a045fd2c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:11.325886 13990 tablet_bootstrap.cc:654] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.327406 13987 tablet_bootstrap.cc:492] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: Bootstrap starting.
I20260807 06:38:11.327487 13865 tablet_service.cc:1467] Processing CreateTablet for tablet 33ffb6e7e93940e5a0c665a045fd2c7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d401e0705adf45fbac920ae0a465d466]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:11.327632 13865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ffb6e7e93940e5a0c665a045fd2c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:11.327797 13987 tablet_bootstrap.cc:654] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.328380 13987 tablet_bootstrap.cc:492] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: No bootstrap required, opened a new log
I20260807 06:38:11.328445 13987 ts_tablet_manager.cc:1397] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:11.328596 13987 raft_consensus.cc:348] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.328681 13987 raft_consensus.cc:374] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.328711 13987 raft_consensus.cc:729] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cefe2c45874ed6a5fae197644e8164, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.328756 13987 consensus_queue.cc:260] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.328876 13987 ts_tablet_manager.cc:1428] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:11.328923 13994 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.328999 13994 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:11.329239 13986 tablet_bootstrap.cc:492] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: Bootstrap starting.
I20260807 06:38:11.329735 13986 tablet_bootstrap.cc:654] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:11.330072 13990 tablet_bootstrap.cc:492] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: No bootstrap required, opened a new log
I20260807 06:38:11.330144 13990 ts_tablet_manager.cc:1397] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 06:38:11.330307 13990 raft_consensus.cc:348] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.330408 13990 raft_consensus.cc:374] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.330430 13990 raft_consensus.cc:729] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8aeacd350241158ad0ec7fc008a3d7, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.330499 13990 consensus_queue.cc:260] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.330574 13986 tablet_bootstrap.cc:492] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: No bootstrap required, opened a new log
I20260807 06:38:11.330592 13990 ts_tablet_manager.cc:1428] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:11.330617 13986 ts_tablet_manager.cc:1397] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:11.330605 13996 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.330696 13996 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:11.330763 13986 raft_consensus.cc:348] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.330830 13986 raft_consensus.cc:374] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:11.330852 13986 raft_consensus.cc:729] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76a28697f4e488c034faf4c14c2c1, State: Initialized, Role: FOLLOWER
I20260807 06:38:11.330899 13986 consensus_queue.cc:260] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.330983 13986 ts_tablet_manager.cc:1428] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:11.331148 13993 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.331239 13993 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:11.409727 13993 raft_consensus.cc:482] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:11.409838 13993 raft_consensus.cc:504] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.410279 13993 leader_election.cc:288] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299), ba8aeacd350241158ad0ec7fc008a3d7 (127.11.114.194:39169)
I20260807 06:38:11.413159 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.410463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:33299 (local address 127.11.114.195:58377)
0807 06:38:11.410537 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:11.410551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:11.410601 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:11.410826 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:11.410827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:11.410829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:11.410908 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.410910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.411574 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.411575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.412337 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.412339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.412893 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.412894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.412972 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.412976 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:11.413042 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:38:11.413082 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 06:38:11.413160 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.410652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:58377 (local address 127.11.114.193:33299)
0807 06:38:11.410700 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.410702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.410710 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:11.410745 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.410746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.410753 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.410821 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:11.410993 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.411481 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.412434 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.412797 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.412893 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.412902 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.413052 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.413090 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:38:11.413417 13495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ffb6e7e93940e5a0c665a045fd2c7f" candidate_uuid: "1ef76a28697f4e488c034faf4c14c2c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81cefe2c45874ed6a5fae197644e8164" is_pre_election: true
I20260807 06:38:11.413520 13495 raft_consensus.cc:2454] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ef76a28697f4e488c034faf4c14c2c1 in term 0.
I20260807 06:38:11.413669 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.410548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:54553 (local address 127.11.114.194:39169)
0807 06:38:11.410605 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.410607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.410717 (+ 110us) server_negotiation.cc:407] Connection header received
0807 06:38:11.410829 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.410830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.410843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.410920 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:11.411094 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.411566 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.412852 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.413245 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.413320 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.413329 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.413518 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.413585 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:38:11.413676 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.410475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:39169 (local address 127.11.114.195:54553)
0807 06:38:11.410627 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:11.410638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:11.410748 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:11.410926 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:11.410927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:11.410929 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:11.411024 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.411025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.411655 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.411656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.412753 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.412755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.413386 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.413387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.413450 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.413453 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:11.413495 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 06:38:11.413581 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260807 06:38:11.413750 13797 leader_election.cc:302] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ef76a28697f4e488c034faf4c14c2c1, 81cefe2c45874ed6a5fae197644e8164; no voters:
I20260807 06:38:11.413885 13993 raft_consensus.cc:2781] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:11.413946 13993 raft_consensus.cc:482] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:11.413971 13993 raft_consensus.cc:3037] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.413991 13695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ffb6e7e93940e5a0c665a045fd2c7f" candidate_uuid: "1ef76a28697f4e488c034faf4c14c2c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" is_pre_election: true
I20260807 06:38:11.414131 13695 raft_consensus.cc:2454] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ef76a28697f4e488c034faf4c14c2c1 in term 0.
I20260807 06:38:11.414786 13993 raft_consensus.cc:504] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.414938 13993 leader_election.cc:288] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [CANDIDATE]: Term 1 election: Requested vote from peers 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299), ba8aeacd350241158ad0ec7fc008a3d7 (127.11.114.194:39169)
I20260807 06:38:11.415118 13495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ffb6e7e93940e5a0c665a045fd2c7f" candidate_uuid: "1ef76a28697f4e488c034faf4c14c2c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81cefe2c45874ed6a5fae197644e8164"
I20260807 06:38:11.415192 13495 raft_consensus.cc:3037] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.415127 13695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ffb6e7e93940e5a0c665a045fd2c7f" candidate_uuid: "1ef76a28697f4e488c034faf4c14c2c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8aeacd350241158ad0ec7fc008a3d7"
I20260807 06:38:11.415225 13695 raft_consensus.cc:3037] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:11.415782 13495 raft_consensus.cc:2454] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ef76a28697f4e488c034faf4c14c2c1 in term 1.
I20260807 06:38:11.415952 13797 leader_election.cc:302] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ef76a28697f4e488c034faf4c14c2c1, 81cefe2c45874ed6a5fae197644e8164; no voters:
I20260807 06:38:11.415971 13695 raft_consensus.cc:2454] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ef76a28697f4e488c034faf4c14c2c1 in term 1.
I20260807 06:38:11.416049 13993 raft_consensus.cc:2781] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:11.416242 13993 raft_consensus.cc:686] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef76a28697f4e488c034faf4c14c2c1, State: Running, Role: LEADER
I20260807 06:38:11.416301 13993 consensus_queue.cc:237] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } }
I20260807 06:38:11.416905 14036 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: TxnStatusTablet state changed. Reason: New leader 1ef76a28697f4e488c034faf4c14c2c1. Latest consensus state: current_term: 1 leader_uuid: "1ef76a28697f4e488c034faf4c14c2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.417006 14036 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:11.417224 14039 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:38:11.417176 13277 catalog_manager.cc:5495] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ef76a28697f4e488c034faf4c14c2c1 (127.11.114.195). New cstate: current_term: 1 leader_uuid: "1ef76a28697f4e488c034faf4c14c2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } health_report { overall_health: HEALTHY } } }
I20260807 06:38:11.417335 14039 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:38:11.425928 14031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.423286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34179 (local address 127.0.0.1:60540)
0807 06:38:11.423385 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:11.423399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:11.423459 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:11.423639 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:11.423640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:11.423641 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:11.423690 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.423692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.424386 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.424387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.424979 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.424981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.425353 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.425354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.425423 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.425444 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:11.425750 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:11.425761 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 06:38:11.425802 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 06:38:11.425852 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 06:38:11.425911 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.423379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.11.114.195:34179)
0807 06:38:11.423421 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.423423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.423435 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:38:11.423536 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.423537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.423555 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.423618 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:11.423765 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.424291 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.425067 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.425274 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.425354 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.425567 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:11.425671 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:11.425752 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.425817 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.425839 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:38:11.427452 13495 raft_consensus.cc:1261] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Refusing update from remote peer 1ef76a28697f4e488c034faf4c14c2c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:11.427512 13695 raft_consensus.cc:1261] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 FOLLOWER]: Refusing update from remote peer 1ef76a28697f4e488c034faf4c14c2c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:11.427726 13993 consensus_queue.cc:1035] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:11.427834 14036 consensus_queue.cc:1035] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:11.428592 13996 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: TxnStatusTablet state changed. Reason: New leader 1ef76a28697f4e488c034faf4c14c2c1. Latest consensus state: current_term: 1 leader_uuid: "1ef76a28697f4e488c034faf4c14c2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.428692 13996 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:11.428645 13994 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: TxnStatusTablet state changed. Reason: New leader 1ef76a28697f4e488c034faf4c14c2c1. Latest consensus state: current_term: 1 leader_uuid: "1ef76a28697f4e488c034faf4c14c2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.428747 13994 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:11.429847 14036 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ef76a28697f4e488c034faf4c14c2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.429931 14036 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:11.429984 14036 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ef76a28697f4e488c034faf4c14c2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.430022 14036 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P 1ef76a28697f4e488c034faf4c14c2c1: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:11.430107 14010 mvcc.cc:205] Tried to move back new op lower bound from 7315802896084475904 to 7315802896041787392. Current Snapshot: MvccSnapshot[applied={T|T < 7315802896084475904 or (T in {7315802896084475904})}]
I20260807 06:38:11.430256 13994 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ef76a28697f4e488c034faf4c14c2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.430338 13994 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P 81cefe2c45874ed6a5fae197644e8164: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:11.430603 13996 tablet_replica.cc:439] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ef76a28697f4e488c034faf4c14c2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 } } peers { permanent_uuid: "ba8aeacd350241158ad0ec7fc008a3d7" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39169 } } peers { permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 } } }
I20260807 06:38:11.430691 13996 tablet_replica.cc:442] T 33ffb6e7e93940e5a0c665a045fd2c7f P ba8aeacd350241158ad0ec7fc008a3d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:11.438235 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.435122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34179 (local address 127.11.114.194:44543)
0807 06:38:11.435289 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:11.435301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:11.435377 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:11.435510 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:11.435510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:11.435512 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:11.435561 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.435562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.436575 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.436576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.437193 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.437195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.437803 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.437804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.437856 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.437873 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:11.438079 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:11.438081 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:38:11.438111 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 06:38:11.438176 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260807 06:38:11.438225 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.435143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:44543 (local address 127.11.114.195:34179)
0807 06:38:11.435193 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.435195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.435371 (+ 176us) server_negotiation.cc:407] Connection header received
0807 06:38:11.435432 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.435433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.435445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.435511 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:11.435607 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.436468 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.437275 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.437638 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.437719 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.437966 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:11.438006 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:11.438080 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.438117 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.438139 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:11.440642 13890 raft_consensus.cc:1261] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Refusing update from remote peer ba8aeacd350241158ad0ec7fc008a3d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:11.440733 13495 raft_consensus.cc:1261] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Refusing update from remote peer ba8aeacd350241158ad0ec7fc008a3d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:11.440948 13996 consensus_queue.cc:1035] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81cefe2c45874ed6a5fae197644e8164" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:11.441108 13999 consensus_queue.cc:1035] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef76a28697f4e488c034faf4c14c2c1" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:11.442378 14003 mvcc.cc:205] Tried to move back new op lower bound from 7315802896139436032 to 7315802895500808192. Current Snapshot: MvccSnapshot[applied={T|T < 7315802896139436032}]
I20260807 06:38:11.450806 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.447775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:41113 (local address 127.11.114.194:39169)
0807 06:38:11.447818 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.447820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.447888 (+ 68us) server_negotiation.cc:407] Connection header received
0807 06:38:11.447937 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.447938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.447949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.448019 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:11.448124 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.448951 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.450094 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.450295 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.450381 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.450554 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:11.450597 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:11.450659 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.450712 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.450735 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:38:11.450821 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.447708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:39169 (local address 127.11.114.195:41113)
0807 06:38:11.447834 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:11.447845 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:11.447896 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:11.448020 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:11.448021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:11.448022 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:11.448080 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.448081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.449041 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.449042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.450003 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:11.450005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.450382 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.450383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:11.450449 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.450471 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:11.450661 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:11.450664 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:11.450711 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 06:38:11.450756 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 06:38:11.459434 13278 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:11.459561 13278 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:11.460281 13278 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a385e686b5d4d11a1fef84a922c1857: Sending DeleteTablet for 3 replicas of tablet 79235d47b93340ba8533080d119ed3b3
I20260807 06:38:11.460584 13669 tablet_service.cc:1514] Processing DeleteTablet for tablet 79235d47b93340ba8533080d119ed3b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:55748
I20260807 06:38:11.460588 13865 tablet_service.cc:1514] Processing DeleteTablet for tablet 79235d47b93340ba8533080d119ed3b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:60532
I20260807 06:38:11.460588 13465 tablet_service.cc:1514] Processing DeleteTablet for tablet 79235d47b93340ba8533080d119ed3b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:42710
I20260807 06:38:11.460696 14015 tablet_replica.cc:330] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: stopping tablet replica
I20260807 06:38:11.460757 14015 raft_consensus.cc:2229] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:11.460793 14015 raft_consensus.cc:2258] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:11.460870 14014 tablet_replica.cc:330] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: stopping tablet replica
I20260807 06:38:11.460932 14014 raft_consensus.cc:2229] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:11.460973 14014 raft_consensus.cc:2258] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:11.461047 14016 tablet_replica.cc:330] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: stopping tablet replica
I20260807 06:38:11.461174 14016 raft_consensus.cc:2229] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:11.461257 14016 raft_consensus.cc:2258] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:11.461635 14015 ts_tablet_manager.cc:1905] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:11.461812 14016 ts_tablet_manager.cc:1905] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:11.463425 14015 ts_tablet_manager.cc:1918] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:11.463425 14016 ts_tablet_manager.cc:1918] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:11.463492 14015 log.cc:1196] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/79235d47b93340ba8533080d119ed3b3
I20260807 06:38:11.463492 14016 log.cc:1196] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/79235d47b93340ba8533080d119ed3b3
I20260807 06:38:11.463723 14015 ts_tablet_manager.cc:1939] T 79235d47b93340ba8533080d119ed3b3 P 1ef76a28697f4e488c034faf4c14c2c1: Deleting consensus metadata
I20260807 06:38:11.463723 14016 ts_tablet_manager.cc:1939] T 79235d47b93340ba8533080d119ed3b3 P ba8aeacd350241158ad0ec7fc008a3d7: Deleting consensus metadata
I20260807 06:38:11.464196 14014 ts_tablet_manager.cc:1905] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:11.464232 13259 catalog_manager.cc:4841] TS ba8aeacd350241158ad0ec7fc008a3d7 (127.11.114.194:39169): tablet 79235d47b93340ba8533080d119ed3b3 (table test-table [id=c83c6be8020349b38967fa79c088d4d1]) successfully deleted
I20260807 06:38:11.464386 13256 catalog_manager.cc:4841] TS 1ef76a28697f4e488c034faf4c14c2c1 (127.11.114.195:34179): tablet 79235d47b93340ba8533080d119ed3b3 (table test-table [id=c83c6be8020349b38967fa79c088d4d1]) successfully deleted
I20260807 06:38:11.465674 14014 ts_tablet_manager.cc:1918] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:11.465744 14014 log.cc:1196] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/79235d47b93340ba8533080d119ed3b3
I20260807 06:38:11.465993 14014 ts_tablet_manager.cc:1939] T 79235d47b93340ba8533080d119ed3b3 P 81cefe2c45874ed6a5fae197644e8164: Deleting consensus metadata
I20260807 06:38:11.466498 13256 catalog_manager.cc:4841] TS 81cefe2c45874ed6a5fae197644e8164 (127.11.114.193:33299): tablet 79235d47b93340ba8533080d119ed3b3 (table test-table [id=c83c6be8020349b38967fa79c088d4d1]) successfully deleted
Aug 07 06:38:11 dist-test-slave-gkw7 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:11.476068 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.465825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.11.114.193:33299)
0807 06:38:11.465866 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.465868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.466255 (+ 387us) server_negotiation.cc:407] Connection header received
0807 06:38:11.466289 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.466290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.466303 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.466366 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:11.467135 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.467607 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.468729 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.468915 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.468973 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.472914 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:11.472926 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:11.472926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:11.472935 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:11.474718 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:11.475149 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:11.475152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:11.475152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:11.475250 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:11.475478 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:11.475479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:11.475479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:11.475647 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:11.475811 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.475877 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.475961 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 06:38:11.476514 13548 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42712
I20260807 06:38:11.484958 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:11.477645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.11.114.254:33169)
0807 06:38:11.477690 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:11.477692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:11.477747 (+ 55us) server_negotiation.cc:407] Connection header received
0807 06:38:11.477824 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:11.477825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:11.477834 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:11.477880 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:11.478639 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.479149 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.480269 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:11.480464 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:11.480522 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:11.482009 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:11.482010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:11.482011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:11.482017 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:11.483740 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:11.484185 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:11.484186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:11.484187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:11.484245 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:11.484479 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:11.484480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:11.484481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:11.484608 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:11.484686 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:11.484833 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 06:38:11.484893 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 06:38:11.485579 13278 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47902
I20260807 06:38:11.486366 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 13370
I20260807 06:38:11.514554 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 13568
I20260807 06:38:11.525709 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 13770
I20260807 06:38:11.536142 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 13235
2026-08-07T06:38:11Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2664 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:11 dist-test-slave-gkw7 krb5kdc[14060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:11 dist-test-slave-gkw7 krb5kdc[14060](info): set up 2 sockets
Aug 07 06:38:11 dist-test-slave-gkw7 krb5kdc[14060](info): commencing operation
krb5kdc: starting...
W20260807 06:38:12.757328 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:12 dist-test-slave-gkw7 krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:12.884234 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:35867
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:35867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:12.965704 14084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:12.965905 14084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:12.965929 14084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:12.967017 14084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:12.967048 14084 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:38:12.967061 14084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:12.967074 14084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:12.967087 14084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:12.967099 14084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:12.968541 14084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:35867
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:35867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:12.968793 14084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:12.968995 14084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:12.974050 14094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:12.974128 14084 server_base.cc:1034] running on GCE node
W20260807 06:38:12.974061 14091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:12.974061 14090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:12.974589 14084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:12.974949 14084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:12.976116 14084 hybrid_clock.cc:648] HybridClock initialized: now 1786084692976092 us; error 58 us; skew 500 ppm
Aug 07 06:38:12 dist-test-slave-gkw7 krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084692, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:12.978693 14084 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:12.979192 14084 webserver.cc:460] Webserver started at http://127.11.114.254:45593/ using document root <none> and password file <none>
I20260807 06:38:12.979425 14084 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:12.979493 14084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:12.979645 14084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:12.980747 14084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "4a8fca9de83946ceb9d7093b42c78e71"
format_stamp: "Formatted at 2026-08-07 06:38:12 on dist-test-slave-gkw7"
server_key: "151f96db65ccbf4a2498c1c480c370c0"
server_key_iv: "804aa6e6b229fb1d99952715d6b91bfa"
server_key_version: "encryptionkey@0"
I20260807 06:38:12.981130 14084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "4a8fca9de83946ceb9d7093b42c78e71"
format_stamp: "Formatted at 2026-08-07 06:38:12 on dist-test-slave-gkw7"
server_key: "151f96db65ccbf4a2498c1c480c370c0"
server_key_iv: "804aa6e6b229fb1d99952715d6b91bfa"
server_key_version: "encryptionkey@0"
I20260807 06:38:12.982646 14084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:12.983484 14108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:12.983695 14084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:12.983805 14084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "4a8fca9de83946ceb9d7093b42c78e71"
format_stamp: "Formatted at 2026-08-07 06:38:12 on dist-test-slave-gkw7"
server_key: "151f96db65ccbf4a2498c1c480c370c0"
server_key_iv: "804aa6e6b229fb1d99952715d6b91bfa"
server_key_version: "encryptionkey@0"
I20260807 06:38:12.983879 14084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:13.001008 14084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:13.003434 14084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:13.003612 14084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:13.008710 14084 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:35867
I20260807 06:38:13.008766 14197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:35867 every 8 connection(s)
I20260807 06:38:13.009133 14084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:13.009434 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 14084
I20260807 06:38:13.009545 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:13.009725 11723 external_mini_cluster.cc:1428] Setting key 3f35bcf14fe695600eb2ebeeaae95aea
I20260807 06:38:13.010011 14198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:13.012871 14198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: Bootstrap starting.
I20260807 06:38:13.013489 14198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:13.013821 14198 log.cc:824] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:13.014530 14198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: No bootstrap required, opened a new log
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:13.016856 14198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8fca9de83946ceb9d7093b42c78e71" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35867 } }
I20260807 06:38:13.017033 14198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:13.017061 14198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a8fca9de83946ceb9d7093b42c78e71, State: Initialized, Role: FOLLOWER
I20260807 06:38:13.017231 14198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8fca9de83946ceb9d7093b42c78e71" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35867 } }
I20260807 06:38:13.017297 14198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:13.017341 14198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:13.017405 14198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:13.018087 14198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8fca9de83946ceb9d7093b42c78e71" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35867 } }
I20260807 06:38:13.018208 14198 leader_election.cc:302] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a8fca9de83946ceb9d7093b42c78e71; no voters:
I20260807 06:38:13.018421 14198 leader_election.cc:288] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:13.018455 14206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:13.018567 14206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [term 1 LEADER]: Becoming Leader. State: Replica: 4a8fca9de83946ceb9d7093b42c78e71, State: Running, Role: LEADER
I20260807 06:38:13.018728 14198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:13.018715 14206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8fca9de83946ceb9d7093b42c78e71" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35867 } }
I20260807 06:38:13.020179 14209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a8fca9de83946ceb9d7093b42c78e71. Latest consensus state: current_term: 1 leader_uuid: "4a8fca9de83946ceb9d7093b42c78e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8fca9de83946ceb9d7093b42c78e71" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35867 } } }
I20260807 06:38:13.020354 14209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:13.020222 14208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a8fca9de83946ceb9d7093b42c78e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8fca9de83946ceb9d7093b42c78e71" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35867 } } }
I20260807 06:38:13.020727 14208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71 [sys.catalog]: This master's current role is: LEADER
W20260807 06:38:13.020934 14225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:38:13.021027 14225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:13.021111 14228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:13.021310 14228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:13.021497 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.010304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.11.114.254:35867)
0807 06:38:13.010499 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.010509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.010541 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:38:13.011097 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.011104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.011153 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.011306 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.011940 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.012848 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.013697 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.014146 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.014270 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.016962 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.016966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.016966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.016977 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.019199 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.019685 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.019687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.019688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.019803 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.020053 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.020054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.020054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.020490 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.020893 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.021088 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.021242 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260807 06:38:13.022985 14228 catalog_manager.cc:1347] Generated new cluster ID: 08121ca119a240eda27d3749b166658e
I20260807 06:38:13.023036 14228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:13.029137 14228 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:13.029891 14228 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:13.036592 14228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: Generated new TSK 0
I20260807 06:38:13.036832 14228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:13.070362 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:35867
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:13.155478 14234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:13.155720 14234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:13.155754 14234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:13.156845 14234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:13.156909 14234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:13.156939 14234 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:38:13.156951 14234 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:13.156968 14234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:13.158435 14234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:35867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:13.158643 14234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:13.158823 14234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:13.163983 14239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:13.163973 14242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:13.164240 14234 server_base.cc:1034] running on GCE node
W20260807 06:38:13.163977 14240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:13.164503 14234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:13.164754 14234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:13.165915 14234 hybrid_clock.cc:648] HybridClock initialized: now 1786084693165891 us; error 46 us; skew 500 ppm
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:13.168404 14234 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:13.168876 14234 webserver.cc:460] Webserver started at http://127.11.114.193:37549/ using document root <none> and password file <none>
I20260807 06:38:13.169130 14234 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:13.169198 14234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:13.169363 14234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:13.170433 14234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "3e69c432863b44dab975eccee216ed15"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "9d455ccdcd7b01b97adf7762de02d1c0"
server_key_iv: "65009127af418a1529087daa33b4419d"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.170801 14234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "3e69c432863b44dab975eccee216ed15"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "9d455ccdcd7b01b97adf7762de02d1c0"
server_key_iv: "65009127af418a1529087daa33b4419d"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.172155 14234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:13.172961 14250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:13.173158 14234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:13.173275 14234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "3e69c432863b44dab975eccee216ed15"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "9d455ccdcd7b01b97adf7762de02d1c0"
server_key_iv: "65009127af418a1529087daa33b4419d"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.173358 14234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:13.188705 14234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:13.191023 14234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:13.191139 14234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:13.191689 14234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:13.191731 14234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:13.191754 14234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:13.191767 14234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:13.199295 14234 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:37533
I20260807 06:38:13.199388 14405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:37533 every 8 connection(s)
I20260807 06:38:13.199744 14234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:13.201443 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.192106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:52225 (local address 127.11.114.254:35867)
0807 06:38:13.192202 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.192204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.192417 (+ 213us) server_negotiation.cc:407] Connection header received
0807 06:38:13.192957 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.192958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.192971 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.193033 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.193817 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.194337 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.195359 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.195667 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.195762 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.198568 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.198570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.198570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.198578 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.200331 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.200695 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.200696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.200696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.200750 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.200944 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.200945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.200945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.201047 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.201098 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.201311 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.201379 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:38:13.201620 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.192098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35867 (local address 127.11.114.193:52225)
0807 06:38:13.192290 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.192332 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.192846 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.193103 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.193107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.193438 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:13.193708 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.193714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.194434 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.194435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.195266 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.195269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.195774 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.195775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.195913 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.196449 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:13.196459 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:13.198470 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:13.200400 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.200404 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.200406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.200626 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.200805 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.200806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.200806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.200893 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.201086 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:13.201089 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:13.201204 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.201282 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.201400 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260807 06:38:13.202140 14406 heartbeater.cc:347] Connected to a master server at 127.11.114.254:35867
I20260807 06:38:13.202226 14406 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:13.202407 14406 heartbeater.cc:511] Master 127.11.114.254:35867 requested a full tablet report, sending...
I20260807 06:38:13.202910 14128 ts_manager.cc:194] Registered new tserver with Master: 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533)
I20260807 06:38:13.203666 14128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:52225
I20260807 06:38:13.205952 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 14234
I20260807 06:38:13.206140 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:13.206363 11723 external_mini_cluster.cc:1428] Setting key b76f76e7e7512b9350f55d48f428fbea
I20260807 06:38:13.209498 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.203281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:54207 (local address 127.11.114.254:35867)
0807 06:38:13.203324 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.203326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.203389 (+ 63us) server_negotiation.cc:407] Connection header received
0807 06:38:13.203460 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.203461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.203471 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.203526 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.203840 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.204332 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.205027 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.205200 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.205251 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.206573 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.206574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.206574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.206582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.208309 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.208639 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.208640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.208641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.208710 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.208897 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.208898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.208898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.209050 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.209128 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.209355 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.209436 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:38:13.209578 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.203202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35867 (local address 127.11.114.193:54207)
0807 06:38:13.203335 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.203346 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.203404 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.203530 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.203531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.203706 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:13.203756 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.203757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.204428 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.204429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.204954 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.204956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.205425 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.205426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.205495 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.206034 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:13.206039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:13.206465 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:13.208398 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.208398 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.208399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.208563 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.208778 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.208778 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.208779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.208832 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.209163 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:13.209164 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:13.209220 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.209305 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.209420 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:13.236999 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:35867
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:13.323092 14416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:13.323294 14416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:13.323319 14416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:13.324443 14416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:13.324486 14416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:13.324507 14416 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:38:13.324519 14416 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:13.324535 14416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:13.326047 14416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:35867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:13.326289 14416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:13.326522 14416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:13.331492 14423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:13.331502 14425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:13.331502 14429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:13.331660 14416 server_base.cc:1034] running on GCE node
I20260807 06:38:13.331851 14416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:13.332091 14416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:13.333262 14416 hybrid_clock.cc:648] HybridClock initialized: now 1786084693333249 us; error 32 us; skew 500 ppm
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:13.335803 14416 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:13.336272 14416 webserver.cc:460] Webserver started at http://127.11.114.194:44595/ using document root <none> and password file <none>
I20260807 06:38:13.336521 14416 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:13.336592 14416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:13.336745 14416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:13.337874 14416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "490edf2139c542a4bd22b5b6a1c81a13"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "1d07134ffb00478f070c44ac65dcb27c"
server_key_iv: "9ae8a6d5832a49cc25f8cbff4adb608b"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.338239 14416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "490edf2139c542a4bd22b5b6a1c81a13"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "1d07134ffb00478f070c44ac65dcb27c"
server_key_iv: "9ae8a6d5832a49cc25f8cbff4adb608b"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.339548 14416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:13.340350 14437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:13.340534 14416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:13.340641 14416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "490edf2139c542a4bd22b5b6a1c81a13"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "1d07134ffb00478f070c44ac65dcb27c"
server_key_iv: "9ae8a6d5832a49cc25f8cbff4adb608b"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.340715 14416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:13.360498 14416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:13.362635 14416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:13.362802 14416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:13.363400 14416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:13.363461 14416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:13.363495 14416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:13.363524 14416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:13.371662 14416 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:43765
I20260807 06:38:13.371797 14598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:43765 every 8 connection(s)
I20260807 06:38:13.372118 14416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:13.372605 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 14416
I20260807 06:38:13.372717 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:13.372926 11723 external_mini_cluster.cc:1428] Setting key 372d3965d12a6da52d266e864ff69856
I20260807 06:38:13.374264 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.363802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:59375 (local address 127.11.114.254:35867)
0807 06:38:13.363860 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.363863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.364100 (+ 237us) server_negotiation.cc:407] Connection header received
0807 06:38:13.364574 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.364576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.364590 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.364664 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.365405 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.365996 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.367051 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.367276 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.367361 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.370646 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.370648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.370648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.370660 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.372544 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.373098 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.373099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.373100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.373176 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.373470 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.373470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.373471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.373664 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.373733 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.374051 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.374164 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:13.374439 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.363796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35867 (local address 127.11.114.194:59375)
0807 06:38:13.363950 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.363989 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.364466 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.364742 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.364745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.365097 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:13.365300 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.365305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.366091 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.366093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.366921 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.366924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.367373 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.367374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.367530 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.368081 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:13.368093 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:13.370523 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:13.372649 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.372652 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.372654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.372997 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.373262 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.373263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.373263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.373404 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.373738 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:13.373746 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:13.373897 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.374013 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.374188 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260807 06:38:13.375010 14599 heartbeater.cc:347] Connected to a master server at 127.11.114.254:35867
I20260807 06:38:13.375113 14599 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:13.375345 14599 heartbeater.cc:511] Master 127.11.114.254:35867 requested a full tablet report, sending...
I20260807 06:38:13.375751 14131 ts_manager.cc:194] Registered new tserver with Master: 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765)
I20260807 06:38:13.376277 14131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:59375
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
I20260807 06:38:13.384202 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.376673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:35021 (local address 127.11.114.254:35867)
0807 06:38:13.376715 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.376717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.377837 (+ 1120us) server_negotiation.cc:407] Connection header received
0807 06:38:13.377897 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.377898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.377906 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.377957 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.378377 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.378852 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.380102 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.380289 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.380335 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.381476 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.381477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.381477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.381483 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.382911 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.383354 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.383355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.383356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.383407 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.383655 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.383656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.383656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.383789 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.383840 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.384022 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.384133 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:38:13.384256 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.376623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35867 (local address 127.11.114.194:35021)
0807 06:38:13.377751 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.377765 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.377837 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.377964 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.377965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.378216 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:13.378291 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.378293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.378975 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.378977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.380007 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.380011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.380366 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.380368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.380479 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.380910 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:13.380916 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:13.381387 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:13.383007 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.383008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.383009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.383267 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.383493 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.383494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.383495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.383588 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.383858 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:13.383859 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:13.383912 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.383998 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.384129 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1095,"thread_start_us":84,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:13.405320 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:35867
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:13.494226 14617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:13.494467 14617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:13.494513 14617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:13.495692 14617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:13.495765 14617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:13.495806 14617 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:38:13.495857 14617 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:13.495899 14617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:13.497421 14617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:35867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:13.497740 14617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:13.498039 14617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:13.503542 14622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:13.503588 14626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:13.503667 14617 server_base.cc:1034] running on GCE node
W20260807 06:38:13.503568 14628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:13.504017 14617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:13.504285 14617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:13.505450 14617 hybrid_clock.cc:648] HybridClock initialized: now 1786084693505441 us; error 33 us; skew 500 ppm
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:13.507938 14617 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:13.508395 14617 webserver.cc:460] Webserver started at http://127.11.114.195:33141/ using document root <none> and password file <none>
I20260807 06:38:13.508651 14617 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:13.508721 14617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:13.508868 14617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:13.509970 14617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "e17590abf34543e7a439cbea449c1bbc"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "0da10ec57b244083fcd7dde899c863a7"
server_key_iv: "7fd967c3febfdbf2241f0f78b141ac01"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.510380 14617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "e17590abf34543e7a439cbea449c1bbc"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "0da10ec57b244083fcd7dde899c863a7"
server_key_iv: "7fd967c3febfdbf2241f0f78b141ac01"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.511713 14617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:13.512460 14637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:13.512661 14617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:13.512769 14617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "e17590abf34543e7a439cbea449c1bbc"
format_stamp: "Formatted at 2026-08-07 06:38:13 on dist-test-slave-gkw7"
server_key: "0da10ec57b244083fcd7dde899c863a7"
server_key_iv: "7fd967c3febfdbf2241f0f78b141ac01"
server_key_version: "encryptionkey@0"
I20260807 06:38:13.512845 14617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:13.542384 14617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:13.545186 14617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:13.545380 14617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:13.546056 14617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:13.546128 14617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:13.546185 14617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:13.546237 14617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:13.554505 14617 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:43163
I20260807 06:38:13.554580 14799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:43163 every 8 connection(s)
I20260807 06:38:13.554927 14617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:38:13.557497 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.546822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:37613 (local address 127.11.114.254:35867)
0807 06:38:13.546879 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.546881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.547179 (+ 298us) server_negotiation.cc:407] Connection header received
0807 06:38:13.547797 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.547800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.547819 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.547897 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.548884 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.549871 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.551002 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.551391 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.551478 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.553994 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.553996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.553996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.554005 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.555832 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.556365 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.556367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.556367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.556439 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.556790 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.556791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.556792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.556929 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.556988 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.557315 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.557395 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:13.557665 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.546785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35867 (local address 127.11.114.195:37613)
0807 06:38:13.547015 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.547082 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.547638 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.548054 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.548059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.548461 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:13.548763 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.548770 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.549998 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.549999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.550926 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.550929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.551516 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.551517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.551673 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.552329 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:13.552338 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:13.553871 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:13.555949 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.555958 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.555960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.556256 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.556516 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.556517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.556517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.556701 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.557005 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:13.557012 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:13.557149 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.557254 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.557391 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":117,"threads_started":1}
I20260807 06:38:13.558245 14800 heartbeater.cc:347] Connected to a master server at 127.11.114.254:35867
I20260807 06:38:13.558344 14800 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:13.558564 14800 heartbeater.cc:511] Master 127.11.114.254:35867 requested a full tablet report, sending...
I20260807 06:38:13.558969 14128 ts_manager.cc:194] Registered new tserver with Master: e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163)
I20260807 06:38:13.559502 14128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:37613
I20260807 06:38:13.561084 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 14617
I20260807 06:38:13.561172 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:13.561368 11723 external_mini_cluster.cc:1428] Setting key 278b24ef510e6aa9d6fdf7c2b3e2498d
I20260807 06:38:13.562439 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:38:13.566393 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.559595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:51483 (local address 127.11.114.254:35867)
0807 06:38:13.559642 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.559644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.559819 (+ 175us) server_negotiation.cc:407] Connection header received
0807 06:38:13.559883 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.559883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.559894 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.559956 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.560313 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.560869 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.562024 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.562313 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.562394 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.563428 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.563430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.563430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.563437 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.565031 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.565471 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.565472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.565472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.565524 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.565848 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.565849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.565850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.565981 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.566054 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.566254 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.566321 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:13.566519 14813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.559524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35867 (local address 127.11.114.195:51483)
0807 06:38:13.559755 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.559767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.559824 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.559975 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.559975 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.560171 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:13.560232 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.560236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.560985 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.560986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.561919 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.561923 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.562423 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.562424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.562506 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.562912 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:13.562917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:13.563338 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:13.565117 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.565119 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.565120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.565391 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.565634 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.565635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.565635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.565724 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.566070 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:13.566071 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:13.566130 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.566218 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.566346 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:13.588001 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.580740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.11.114.254:35867)
0807 06:38:13.580806 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.580808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.580818 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:13.580849 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.580850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.580858 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.580911 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.581588 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.582156 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.582900 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.583115 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.583175 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.585353 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.585354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.585355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.585361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.586914 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.587253 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.587254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.587254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.587320 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.587543 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.587544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.587544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.587685 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.587771 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.587871 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.587934 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:38:13.589730 14128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:13.591388 14128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:13.597967 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.593878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:43163 (local address 127.0.0.1:37970)
0807 06:38:13.594063 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.594108 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.594179 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.594609 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.594612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.594614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:13.594683 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.594685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.595649 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.595650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.596768 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.596771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.597669 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.597669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.597748 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.597761 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.597832 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.597886 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
I20260807 06:38:13.597985 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.593989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.11.114.195:43163)
0807 06:38:13.594192 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.594198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.594209 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:38:13.594276 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.594278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.594293 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.594372 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:13.594786 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.595565 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.596869 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.597592 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.597670 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.597729 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.597857 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.597895 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":91,"threads_started":1}
I20260807 06:38:13.598380 14826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.593878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43765 (local address 127.0.0.1:50764)
0807 06:38:13.594281 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.594293 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.594346 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.594704 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.594705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.594706 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:13.594780 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.594782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.596102 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.596103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.597176 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.597178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.598142 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.598143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.598219 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.598223 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.598269 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.598307 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":20,"thread_start_us":90,"threads_started":1}
I20260807 06:38:13.598587 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.594154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:37533 (local address 127.0.0.1:48374)
0807 06:38:13.594465 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.594476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.594523 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.594842 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.594843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.594845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:13.594898 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.594899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.596134 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.596135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.597020 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.597022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.598355 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.598356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.598424 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.598429 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.598484 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.598528 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":94,"threads_started":1}
I20260807 06:38:13.598724 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.594102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.11.114.194:43765)
0807 06:38:13.594429 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.594433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.594442 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:13.594485 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.594487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.594498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.594596 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:13.594861 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.596004 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.597257 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.597987 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.598050 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.598102 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.598547 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.598606 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":85,"threads_started":1}
I20260807 06:38:13.598708 14710 tablet_service.cc:1467] Processing CreateTablet for tablet 10359d2d7cdd42aa9d531c956ce0c4f8 (DEFAULT_TABLE table=test-table [id=6f26ff613a164fd8ae656d82d9e11839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:13.598922 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.594426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.11.114.193:37533)
0807 06:38:13.594631 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.594636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.594645 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:13.594683 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.594685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.594701 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.594776 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:13.594978 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.596045 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.597112 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.597876 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.597950 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.598001 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.598749 (+ 748us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.598796 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":99,"threads_started":1}
I20260807 06:38:13.599119 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10359d2d7cdd42aa9d531c956ce0c4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:13.599325 14507 tablet_service.cc:1467] Processing CreateTablet for tablet 10359d2d7cdd42aa9d531c956ce0c4f8 (DEFAULT_TABLE table=test-table [id=6f26ff613a164fd8ae656d82d9e11839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:13.599643 14507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10359d2d7cdd42aa9d531c956ce0c4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:13.601882 14833 tablet_bootstrap.cc:492] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Bootstrap starting.
I20260807 06:38:13.602311 14320 tablet_service.cc:1467] Processing CreateTablet for tablet 10359d2d7cdd42aa9d531c956ce0c4f8 (DEFAULT_TABLE table=test-table [id=6f26ff613a164fd8ae656d82d9e11839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:13.602608 14834 tablet_bootstrap.cc:492] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Bootstrap starting.
I20260807 06:38:13.602689 14320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10359d2d7cdd42aa9d531c956ce0c4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:13.602991 14833 tablet_bootstrap.cc:654] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:13.603273 14834 tablet_bootstrap.cc:654] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:13.603354 14833 log.cc:824] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:13.603614 14834 log.cc:824] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:13.604205 14833 tablet_bootstrap.cc:492] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: No bootstrap required, opened a new log
I20260807 06:38:13.604288 14834 tablet_bootstrap.cc:492] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: No bootstrap required, opened a new log
I20260807 06:38:13.604374 14834 ts_tablet_manager.cc:1397] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:13.604466 14833 ts_tablet_manager.cc:1397] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:38:13.605926 14834 raft_consensus.cc:348] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.606086 14834 raft_consensus.cc:374] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:13.606132 14834 raft_consensus.cc:729] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490edf2139c542a4bd22b5b6a1c81a13, State: Initialized, Role: FOLLOWER
I20260807 06:38:13.606289 14834 consensus_queue.cc:260] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.606407 14837 tablet_bootstrap.cc:492] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Bootstrap starting.
I20260807 06:38:13.606608 14834 ts_tablet_manager.cc:1428] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:38:13.606745 14599 heartbeater.cc:503] Master 127.11.114.254:35867 was elected leader, sending a full tablet report...
I20260807 06:38:13.606652 14833 raft_consensus.cc:348] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.606804 14833 raft_consensus.cc:374] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:13.606840 14833 raft_consensus.cc:729] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17590abf34543e7a439cbea449c1bbc, State: Initialized, Role: FOLLOWER
I20260807 06:38:13.606945 14833 consensus_queue.cc:260] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.607210 14837 tablet_bootstrap.cc:654] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:13.607223 14833 ts_tablet_manager.cc:1428] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:38:13.607246 14800 heartbeater.cc:503] Master 127.11.114.254:35867 was elected leader, sending a full tablet report...
I20260807 06:38:13.607561 14837 log.cc:824] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:13.608417 14837 tablet_bootstrap.cc:492] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: No bootstrap required, opened a new log
I20260807 06:38:13.608474 14837 ts_tablet_manager.cc:1397] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:13.609843 14837 raft_consensus.cc:348] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.609959 14837 raft_consensus.cc:374] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:13.609980 14837 raft_consensus.cc:729] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e69c432863b44dab975eccee216ed15, State: Initialized, Role: FOLLOWER
I20260807 06:38:13.610116 14837 consensus_queue.cc:260] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.610342 14837 ts_tablet_manager.cc:1428] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:38:13.610460 14406 heartbeater.cc:503] Master 127.11.114.254:35867 was elected leader, sending a full tablet report...
W20260807 06:38:13.623425 14600 tablet.cc:2369] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:38:13.700500 14407 tablet.cc:2369] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:13.745709 14841 raft_consensus.cc:482] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:13.745884 14841 raft_consensus.cc:504] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.746336 14841 leader_election.cc:288] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163), 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765)
I20260807 06:38:13.749526 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.746804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:33507 (local address 127.11.114.195:43163)
0807 06:38:13.746870 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.746872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.746905 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:38:13.746934 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.746935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.746943 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.746981 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:13.747178 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.747704 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.748789 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.749152 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.749236 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.749246 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.749402 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.749451 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:38:13.749600 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.746583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43765 (local address 127.11.114.193:53029)
0807 06:38:13.746755 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.746767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.746830 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.747055 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.747056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.747058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:13.747151 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.747153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.747863 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.747863 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.748654 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.748657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.749277 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.749278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.749354 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.749357 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.749431 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.749469 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":48,"thread_start_us":105,"threads_started":1}
I20260807 06:38:13.749599 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.746703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:53029 (local address 127.11.114.194:43765)
0807 06:38:13.746756 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.746759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.746812 (+ 53us) server_negotiation.cc:407] Connection header received
0807 06:38:13.746927 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.746929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.746940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.747038 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:13.747278 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.747765 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.748761 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.749157 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.749257 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.749266 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.749458 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.749503 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:38:13.749847 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.746628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:43163 (local address 127.11.114.193:33507)
0807 06:38:13.746787 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.746800 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.746864 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.747017 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.747018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.747020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:13.747093 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.747096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.747798 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.747799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.748662 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.748665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.749245 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.749246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.749328 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.749334 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.749383 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.749450 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":4,"thread_start_us":83,"threads_started":1}
I20260807 06:38:13.749930 14536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10359d2d7cdd42aa9d531c956ce0c4f8" candidate_uuid: "3e69c432863b44dab975eccee216ed15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490edf2139c542a4bd22b5b6a1c81a13" is_pre_election: true
I20260807 06:38:13.750149 14536 raft_consensus.cc:2454] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e69c432863b44dab975eccee216ed15 in term 0.
I20260807 06:38:13.750188 14742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10359d2d7cdd42aa9d531c956ce0c4f8" candidate_uuid: "3e69c432863b44dab975eccee216ed15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17590abf34543e7a439cbea449c1bbc" is_pre_election: true
I20260807 06:38:13.750349 14742 raft_consensus.cc:2454] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e69c432863b44dab975eccee216ed15 in term 0.
I20260807 06:38:13.750402 14252 leader_election.cc:302] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e69c432863b44dab975eccee216ed15, 490edf2139c542a4bd22b5b6a1c81a13; no voters:
I20260807 06:38:13.750511 14841 raft_consensus.cc:2781] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:13.750563 14841 raft_consensus.cc:482] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:13.750617 14841 raft_consensus.cc:3037] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:13.751362 14841 raft_consensus.cc:504] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.751499 14841 leader_election.cc:288] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [CANDIDATE]: Term 1 election: Requested vote from peers e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163), 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765)
I20260807 06:38:13.751680 14742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10359d2d7cdd42aa9d531c956ce0c4f8" candidate_uuid: "3e69c432863b44dab975eccee216ed15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17590abf34543e7a439cbea449c1bbc"
I20260807 06:38:13.751667 14536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10359d2d7cdd42aa9d531c956ce0c4f8" candidate_uuid: "3e69c432863b44dab975eccee216ed15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490edf2139c542a4bd22b5b6a1c81a13"
I20260807 06:38:13.751773 14742 raft_consensus.cc:3037] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:13.751757 14536 raft_consensus.cc:3037] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:13.752511 14742 raft_consensus.cc:2454] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e69c432863b44dab975eccee216ed15 in term 1.
I20260807 06:38:13.752642 14536 raft_consensus.cc:2454] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e69c432863b44dab975eccee216ed15 in term 1.
I20260807 06:38:13.752758 14253 leader_election.cc:302] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e69c432863b44dab975eccee216ed15, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:13.752887 14841 raft_consensus.cc:2781] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:13.753021 14841 raft_consensus.cc:686] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 1 LEADER]: Becoming Leader. State: Replica: 3e69c432863b44dab975eccee216ed15, State: Running, Role: LEADER
I20260807 06:38:13.753151 14841 consensus_queue.cc:237] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } }
I20260807 06:38:13.754043 14128 catalog_manager.cc:5495] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e69c432863b44dab975eccee216ed15 (127.11.114.193). New cstate: current_term: 1 leader_uuid: "3e69c432863b44dab975eccee216ed15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } health_report { overall_health: UNKNOWN } } }
W20260807 06:38:13.805670 14801 tablet.cc:2369] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 06:38:13 dist-test-slave-gkw7 krb5kdc[14060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084692, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:13.826246 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.818419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35867 (local address 127.0.0.1:45850)
0807 06:38:13.818720 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:13.818732 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:13.818807 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:13.818991 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:13.818991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:13.819454 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:13.819512 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.819515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.820252 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.820253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.820892 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:13.820894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.821283 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.821284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:13.821347 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.822016 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:13.822021 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:13.823552 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:13.825113 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.825121 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.825123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.825346 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.825613 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:13.825614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:13.825614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:13.825679 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:13.825908 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:13.825915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:13.825963 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 06:38:13.826017 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 06:38:13.826150 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":229,"threads_started":1}
I20260807 06:38:13.826303 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:13.818461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.11.114.254:35867)
0807 06:38:13.818568 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:13.818571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:13.818859 (+ 288us) server_negotiation.cc:407] Connection header received
0807 06:38:13.818893 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:13.818894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:13.818905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:13.818974 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:13.819598 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.820137 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.820973 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:13.821187 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:13.821249 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:13.823656 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:13.823657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:13.823658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:13.823664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:13.825008 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.825413 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.825414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.825414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.825470 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:13.825734 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:13.825735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:13.825735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:13.825845 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:13.825900 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:13.826048 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 06:38:13.826199 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":71}
I20260807 06:38:13.828341 14128 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:45850:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:38:13.828523 14128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:13.830148 14507 tablet_service.cc:1467] Processing CreateTablet for tablet 54c9d9b2df8443f0a437d09bfe3895a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f1afbc7700a4f668dbd8da9bcabbefa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:13.830299 14507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54c9d9b2df8443f0a437d09bfe3895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:13.830323 14710 tablet_service.cc:1467] Processing CreateTablet for tablet 54c9d9b2df8443f0a437d09bfe3895a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f1afbc7700a4f668dbd8da9bcabbefa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:13.830564 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54c9d9b2df8443f0a437d09bfe3895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:13.831676 14834 tablet_bootstrap.cc:492] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: Bootstrap starting.
I20260807 06:38:13.831841 14320 tablet_service.cc:1467] Processing CreateTablet for tablet 54c9d9b2df8443f0a437d09bfe3895a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f1afbc7700a4f668dbd8da9bcabbefa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:13.832033 14320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54c9d9b2df8443f0a437d09bfe3895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:13.832201 14833 tablet_bootstrap.cc:492] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: Bootstrap starting.
I20260807 06:38:13.832403 14834 tablet_bootstrap.cc:654] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:13.832813 14833 tablet_bootstrap.cc:654] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:13.833295 14834 tablet_bootstrap.cc:492] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: No bootstrap required, opened a new log
I20260807 06:38:13.833349 14834 ts_tablet_manager.cc:1397] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:13.833505 14834 raft_consensus.cc:348] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:13.833614 14834 raft_consensus.cc:374] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:13.833636 14834 raft_consensus.cc:729] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490edf2139c542a4bd22b5b6a1c81a13, State: Initialized, Role: FOLLOWER
I20260807 06:38:13.833683 14834 consensus_queue.cc:260] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:13.833735 14833 tablet_bootstrap.cc:492] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: No bootstrap required, opened a new log
I20260807 06:38:13.833772 14837 tablet_bootstrap.cc:492] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: Bootstrap starting.
I20260807 06:38:13.833788 14833 ts_tablet_manager.cc:1397] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:38:13.833796 14834 ts_tablet_manager.cc:1428] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:13.833928 14838 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:13.833961 14833 raft_consensus.cc:348] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:13.834003 14838 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:13.834020 14833 raft_consensus.cc:374] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:13.834040 14833 raft_consensus.cc:729] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17590abf34543e7a439cbea449c1bbc, State: Initialized, Role: FOLLOWER
I20260807 06:38:13.834100 14833 consensus_queue.cc:260] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:13.834200 14837 tablet_bootstrap.cc:654] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:13.834218 14833 ts_tablet_manager.cc:1428] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:13.834354 14839 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:13.834439 14839 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:13.835094 14837 tablet_bootstrap.cc:492] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: No bootstrap required, opened a new log
I20260807 06:38:13.835158 14837 ts_tablet_manager.cc:1397] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:13.835302 14837 raft_consensus.cc:348] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:13.835373 14837 raft_consensus.cc:374] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:13.835405 14837 raft_consensus.cc:729] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e69c432863b44dab975eccee216ed15, State: Initialized, Role: FOLLOWER
I20260807 06:38:13.835459 14837 consensus_queue.cc:260] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:13.835543 14837 ts_tablet_manager.cc:1428] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:13.835603 14841 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:13.835732 14841 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:14.054864 14838 raft_consensus.cc:482] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:14.054988 14838 raft_consensus.cc:504] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.055394 14838 leader_election.cc:288] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533), e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163)
I20260807 06:38:14.059578 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.056107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:47877 (local address 127.11.114.195:43163)
0807 06:38:14.056160 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.056163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.056172 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:14.056203 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.056204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.056214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.056280 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:14.056628 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.057340 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.058595 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.059118 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.059232 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.059249 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.059442 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.059487 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 06:38:14.059607 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.055930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:43163 (local address 127.11.114.194:47877)
0807 06:38:14.056058 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.056069 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.056123 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.056333 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.056334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.056337 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:14.056492 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.056494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.057423 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.057424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.058488 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.058491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.059236 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.059238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.059330 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.059336 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.059407 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.059461 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":53,"threads_started":1}
I20260807 06:38:14.059700 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.055824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:45915 (local address 127.11.114.193:37533)
0807 06:38:14.055891 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.055896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.055936 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:38:14.056024 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.056025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.056056 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.056144 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:14.056443 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.057145 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.058698 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.059249 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.059338 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.059348 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.059564 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.059606 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:38:14.059795 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.055699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:37533 (local address 127.11.114.194:45915)
0807 06:38:14.055851 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.055865 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.055944 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.056221 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.056223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.056227 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:14.056322 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.056325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.057245 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.057247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.058604 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.058607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.059346 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.059347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.059473 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.059477 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.059543 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.059599 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":18,"thread_start_us":90,"threads_started":1}
I20260807 06:38:14.059926 14742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54c9d9b2df8443f0a437d09bfe3895a7" candidate_uuid: "490edf2139c542a4bd22b5b6a1c81a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17590abf34543e7a439cbea449c1bbc" is_pre_election: true
I20260807 06:38:14.060024 14742 raft_consensus.cc:2454] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 490edf2139c542a4bd22b5b6a1c81a13 in term 0.
I20260807 06:38:14.060174 14347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54c9d9b2df8443f0a437d09bfe3895a7" candidate_uuid: "490edf2139c542a4bd22b5b6a1c81a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e69c432863b44dab975eccee216ed15" is_pre_election: true
I20260807 06:38:14.060235 14441 leader_election.cc:302] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490edf2139c542a4bd22b5b6a1c81a13, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:14.060310 14347 raft_consensus.cc:2454] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 490edf2139c542a4bd22b5b6a1c81a13 in term 0.
I20260807 06:38:14.060407 14838 raft_consensus.cc:2781] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:14.060456 14838 raft_consensus.cc:482] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:14.060483 14838 raft_consensus.cc:3037] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:14.061215 14838 raft_consensus.cc:504] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.061378 14838 leader_election.cc:288] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [CANDIDATE]: Term 1 election: Requested vote from peers 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533), e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163)
I20260807 06:38:14.061612 14347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54c9d9b2df8443f0a437d09bfe3895a7" candidate_uuid: "490edf2139c542a4bd22b5b6a1c81a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e69c432863b44dab975eccee216ed15"
I20260807 06:38:14.061612 14742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54c9d9b2df8443f0a437d09bfe3895a7" candidate_uuid: "490edf2139c542a4bd22b5b6a1c81a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17590abf34543e7a439cbea449c1bbc"
I20260807 06:38:14.061713 14742 raft_consensus.cc:3037] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:14.061713 14347 raft_consensus.cc:3037] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:14.062677 14347 raft_consensus.cc:2454] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 490edf2139c542a4bd22b5b6a1c81a13 in term 1.
I20260807 06:38:14.062677 14742 raft_consensus.cc:2454] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 490edf2139c542a4bd22b5b6a1c81a13 in term 1.
I20260807 06:38:14.062876 14441 leader_election.cc:302] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490edf2139c542a4bd22b5b6a1c81a13, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:14.062984 14838 raft_consensus.cc:2781] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:14.063148 14838 raft_consensus.cc:686] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 LEADER]: Becoming Leader. State: Replica: 490edf2139c542a4bd22b5b6a1c81a13, State: Running, Role: LEADER
I20260807 06:38:14.063259 14838 consensus_queue.cc:237] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.063807 14864 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: TxnStatusTablet state changed. Reason: New leader 490edf2139c542a4bd22b5b6a1c81a13. Latest consensus state: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:14.063932 14864 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:14.064160 14866 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:38:14.064307 14866 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:38:14.064222 14131 catalog_manager.cc:5495] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } health_report { overall_health: UNKNOWN } } }
I20260807 06:38:14.119256 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.116365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.11.114.194:43765)
0807 06:38:14.116419 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.116422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.116432 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:14.116537 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.116538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.116566 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.116658 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:14.116808 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.117342 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.118274 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.118519 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.118618 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.118819 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.118933 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.119029 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.119139 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.119168 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:14.119289 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.116277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43765 (local address 127.0.0.1:50776)
0807 06:38:14.116358 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.116371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.116439 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.116665 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.116665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.116667 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:14.116746 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.116748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.117432 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.117433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.118156 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.118158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.118618 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.118620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.118696 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.118717 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.119017 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.119027 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.119110 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.119168 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 06:38:14.121311 14347 raft_consensus.cc:1261] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Refusing update from remote peer 490edf2139c542a4bd22b5b6a1c81a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:14.121353 14742 raft_consensus.cc:1261] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Refusing update from remote peer 490edf2139c542a4bd22b5b6a1c81a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:14.121646 14864 consensus_queue.cc:1035] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:14.121891 14838 consensus_queue.cc:1035] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:14.123086 14841 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: TxnStatusTablet state changed. Reason: New leader 490edf2139c542a4bd22b5b6a1c81a13. Latest consensus state: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:14.123193 14841 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:14.123138 14839 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: TxnStatusTablet state changed. Reason: New leader 490edf2139c542a4bd22b5b6a1c81a13. Latest consensus state: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:14.123240 14839 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:14.124404 14841 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:14.124531 14841 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P 3e69c432863b44dab975eccee216ed15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:14.124485 14838 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:14.124672 14838 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:14.125253 14871 mvcc.cc:205] Tried to move back new op lower bound from 7315802907117993984 to 7315802906883764224. Current Snapshot: MvccSnapshot[applied={T|T < 7315802907117993984}]
I20260807 06:38:14.124486 14864 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:14.125607 14864 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P 490edf2139c542a4bd22b5b6a1c81a13: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:14.125752 14839 tablet_replica.cc:439] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } } }
I20260807 06:38:14.125849 14839 tablet_replica.cc:442] T 54c9d9b2df8443f0a437d09bfe3895a7 P e17590abf34543e7a439cbea449c1bbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:14.126189 14872 mvcc.cc:205] Tried to move back new op lower bound from 7315802907117993984 to 7315802906883764224. Current Snapshot: MvccSnapshot[applied={T|T < 7315802907117993984}]
I20260807 06:38:14.131045 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.127692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.11.114.193:37533)
0807 06:38:14.127760 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.127762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.127860 (+ 98us) server_negotiation.cc:407] Connection header received
0807 06:38:14.127953 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.127953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.127970 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.128030 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:14.128242 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.128878 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.130164 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.130379 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.130439 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.130698 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.130797 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.130867 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.130970 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.130989 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:38:14.137110 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.134056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:45249 (local address 127.11.114.194:43765)
0807 06:38:14.134138 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.134140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.134235 (+ 95us) server_negotiation.cc:407] Connection header received
0807 06:38:14.134289 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.134291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.134305 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.134378 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:14.134491 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.135265 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.136223 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.136556 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.136642 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.136832 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.136870 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.136940 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.137006 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.137031 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:38:14.137135 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.133954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43765 (local address 127.11.114.193:45249)
0807 06:38:14.134146 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.134157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.134227 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.134377 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.134377 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.134378 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:14.134432 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.134434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.135420 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.135421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.136121 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.136124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.136651 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.136652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.136709 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.136723 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.136947 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.136948 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.136984 (+ 36us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.137023 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
I20260807 06:38:14.139967 14742 raft_consensus.cc:1261] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Refusing update from remote peer 3e69c432863b44dab975eccee216ed15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:14.139998 14536 raft_consensus.cc:1261] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Refusing update from remote peer 3e69c432863b44dab975eccee216ed15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:14.140287 14841 consensus_queue.cc:1035] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:14.140519 14845 consensus_queue.cc:1035] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:14.150590 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.147328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:53643 (local address 127.11.114.193:37533)
0807 06:38:14.147397 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.147399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.147409 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:14.147480 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.147481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.147490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.147539 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:14.147675 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.148173 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.149460 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.149838 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.149918 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.150213 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.150249 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.150343 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.150473 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.150504 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:38:14.150637 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.147126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:37533 (local address 127.11.114.194:53643)
0807 06:38:14.147319 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.147330 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.147416 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.147541 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.147541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.147542 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:14.147605 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.147607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.148272 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.148273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.149352 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.149355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.149945 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.149946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.150091 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.150110 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.150369 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.150371 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.150450 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.150501 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260807 06:38:14.160441 14128 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:14.160585 14128 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:14.161461 14128 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: Sending DeleteTablet for 3 replicas of tablet 10359d2d7cdd42aa9d531c956ce0c4f8
I20260807 06:38:14.161935 14710 tablet_service.cc:1514] Processing DeleteTablet for tablet 10359d2d7cdd42aa9d531c956ce0c4f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:14 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260807 06:38:14.162081 14507 tablet_service.cc:1514] Processing DeleteTablet for tablet 10359d2d7cdd42aa9d531c956ce0c4f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:14 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:50764
I20260807 06:38:14.162150 14320 tablet_service.cc:1514] Processing DeleteTablet for tablet 10359d2d7cdd42aa9d531c956ce0c4f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:14 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:48374
I20260807 06:38:14.162400 14910 tablet_replica.cc:330] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: stopping tablet replica
I20260807 06:38:14.162539 14910 raft_consensus.cc:2229] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:14.162647 14910 raft_consensus.cc:2258] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:14.162710 14128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:14.162855 14128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:14.163100 14910 ts_tablet_manager.cc:1905] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:14.164618 14507 tablet_service.cc:1467] Processing CreateTablet for tablet bfd801fa29284d2293f8c93932ee7d82 (DEFAULT_TABLE table=test-table [id=9914a1da4dab465698e630c63eb87b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:14.164749 14507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfd801fa29284d2293f8c93932ee7d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:14.165185 14910 ts_tablet_manager.cc:1918] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:14.165266 14910 log.cc:1196] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/10359d2d7cdd42aa9d531c956ce0c4f8
I20260807 06:38:14.165669 14910 ts_tablet_manager.cc:1939] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 3e69c432863b44dab975eccee216ed15: Deleting consensus metadata
I20260807 06:38:14.166124 14113 catalog_manager.cc:4841] TS 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533): tablet 10359d2d7cdd42aa9d531c956ce0c4f8 (table test-table [id=6f26ff613a164fd8ae656d82d9e11839]) successfully deleted
I20260807 06:38:14.166294 14834 tablet_bootstrap.cc:492] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: Bootstrap starting.
I20260807 06:38:14.166451 14320 tablet_service.cc:1467] Processing CreateTablet for tablet bfd801fa29284d2293f8c93932ee7d82 (DEFAULT_TABLE table=test-table [id=9914a1da4dab465698e630c63eb87b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:14.166574 14320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfd801fa29284d2293f8c93932ee7d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:14.166772 14834 tablet_bootstrap.cc:654] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:14.167725 14834 tablet_bootstrap.cc:492] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: No bootstrap required, opened a new log
I20260807 06:38:14.167778 14834 ts_tablet_manager.cc:1397] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:14.167943 14837 tablet_bootstrap.cc:492] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: Bootstrap starting.
I20260807 06:38:14.167940 14834 raft_consensus.cc:348] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.168017 14834 raft_consensus.cc:374] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:14.168045 14834 raft_consensus.cc:729] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490edf2139c542a4bd22b5b6a1c81a13, State: Initialized, Role: FOLLOWER
I20260807 06:38:14.168102 14834 consensus_queue.cc:260] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.168208 14834 ts_tablet_manager.cc:1428] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:14.168383 14837 tablet_bootstrap.cc:654] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:14.169382 14710 tablet_service.cc:1467] Processing CreateTablet for tablet bfd801fa29284d2293f8c93932ee7d82 (DEFAULT_TABLE table=test-table [id=9914a1da4dab465698e630c63eb87b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:14.169512 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfd801fa29284d2293f8c93932ee7d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:14.170985 14833 tablet_bootstrap.cc:492] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: Bootstrap starting.
I20260807 06:38:14.171468 14833 tablet_bootstrap.cc:654] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:14.171525 14837 tablet_bootstrap.cc:492] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: No bootstrap required, opened a new log
I20260807 06:38:14.171584 14837 ts_tablet_manager.cc:1397] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 06:38:14.171726 14837 raft_consensus.cc:348] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.171789 14837 raft_consensus.cc:374] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:14.171813 14837 raft_consensus.cc:729] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e69c432863b44dab975eccee216ed15, State: Initialized, Role: FOLLOWER
I20260807 06:38:14.171860 14837 consensus_queue.cc:260] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.171952 14837 ts_tablet_manager.cc:1428] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:14.172389 14907 tablet_replica.cc:330] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: stopping tablet replica
I20260807 06:38:14.172564 14907 raft_consensus.cc:2229] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:14.172691 14907 raft_consensus.cc:2258] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:14.173163 14907 ts_tablet_manager.cc:1905] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:14.173622 14909 tablet_replica.cc:330] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: stopping tablet replica
I20260807 06:38:14.173745 14909 raft_consensus.cc:2229] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:14.173825 14909 raft_consensus.cc:2258] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:14.174227 14909 ts_tablet_manager.cc:1905] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:14.174597 14833 tablet_bootstrap.cc:492] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: No bootstrap required, opened a new log
I20260807 06:38:14.174669 14833 ts_tablet_manager.cc:1397] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 06:38:14.174868 14833 raft_consensus.cc:348] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.174947 14833 raft_consensus.cc:374] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:14.174973 14833 raft_consensus.cc:729] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17590abf34543e7a439cbea449c1bbc, State: Initialized, Role: FOLLOWER
I20260807 06:38:14.175036 14833 consensus_queue.cc:260] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.175144 14833 ts_tablet_manager.cc:1428] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:14.175129 14907 ts_tablet_manager.cc:1918] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:14.175276 14907 log.cc:1196] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/10359d2d7cdd42aa9d531c956ce0c4f8
I20260807 06:38:14.175638 14907 ts_tablet_manager.cc:1939] T 10359d2d7cdd42aa9d531c956ce0c4f8 P e17590abf34543e7a439cbea449c1bbc: Deleting consensus metadata
I20260807 06:38:14.176126 14113 catalog_manager.cc:4841] TS e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163): tablet 10359d2d7cdd42aa9d531c956ce0c4f8 (table test-table [id=6f26ff613a164fd8ae656d82d9e11839]) successfully deleted
I20260807 06:38:14.176631 14909 ts_tablet_manager.cc:1918] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:14.176697 14909 log.cc:1196] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/10359d2d7cdd42aa9d531c956ce0c4f8
I20260807 06:38:14.177018 14909 ts_tablet_manager.cc:1939] T 10359d2d7cdd42aa9d531c956ce0c4f8 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting consensus metadata
I20260807 06:38:14.177620 14112 catalog_manager.cc:4841] TS 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765): tablet 10359d2d7cdd42aa9d531c956ce0c4f8 (table test-table [id=6f26ff613a164fd8ae656d82d9e11839]) successfully deleted
I20260807 06:38:14.324896 14839 raft_consensus.cc:482] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:14.325022 14839 raft_consensus.cc:504] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.325413 14839 leader_election.cc:288] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533), 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765)
I20260807 06:38:14.328673 14915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.325662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:37533 (local address 127.11.114.195:41619)
0807 06:38:14.325857 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.325870 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.325980 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.326161 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.326162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.326164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:14.326250 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.326252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.327068 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.327070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.327841 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.327843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.328389 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.328390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.328471 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.328475 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.328524 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.328560 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":2,"thread_start_us":122,"threads_started":1}
I20260807 06:38:14.328673 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.325756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:41619 (local address 127.11.114.193:37533)
0807 06:38:14.325830 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.325833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.325955 (+ 122us) server_negotiation.cc:407] Connection header received
0807 06:38:14.326057 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.326058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.326073 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.326153 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:14.326329 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.326969 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.327937 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.328302 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.328364 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.328372 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.328559 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.328602 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:38:14.328748 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.325813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:43307 (local address 127.11.114.194:43765)
0807 06:38:14.325860 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.325863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.325984 (+ 121us) server_negotiation.cc:407] Connection header received
0807 06:38:14.326072 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.326073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.326083 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.326162 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:14.326378 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.326840 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.327946 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.328328 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.328421 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.328429 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.328628 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.328670 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:38:14.328809 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.325677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43765 (local address 127.11.114.195:43307)
0807 06:38:14.325869 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.325880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.325979 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.326187 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.326188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.326191 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:14.326275 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.326277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.326927 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.326928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.327839 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.327842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.328440 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.328441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.328538 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.328545 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.328610 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.328649 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260807 06:38:14.329042 14347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfd801fa29284d2293f8c93932ee7d82" candidate_uuid: "e17590abf34543e7a439cbea449c1bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e69c432863b44dab975eccee216ed15" is_pre_election: true
I20260807 06:38:14.329056 14536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfd801fa29284d2293f8c93932ee7d82" candidate_uuid: "e17590abf34543e7a439cbea449c1bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490edf2139c542a4bd22b5b6a1c81a13" is_pre_election: true
I20260807 06:38:14.329144 14347 raft_consensus.cc:2454] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17590abf34543e7a439cbea449c1bbc in term 0.
I20260807 06:38:14.329157 14536 raft_consensus.cc:2454] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17590abf34543e7a439cbea449c1bbc in term 0.
I20260807 06:38:14.329327 14641 leader_election.cc:302] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e69c432863b44dab975eccee216ed15, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:14.329466 14839 raft_consensus.cc:2781] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:14.329507 14839 raft_consensus.cc:482] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:14.329563 14839 raft_consensus.cc:3037] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:14.330273 14839 raft_consensus.cc:504] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.330410 14839 leader_election.cc:288] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [CANDIDATE]: Term 1 election: Requested vote from peers 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533), 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765)
I20260807 06:38:14.330571 14347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfd801fa29284d2293f8c93932ee7d82" candidate_uuid: "e17590abf34543e7a439cbea449c1bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e69c432863b44dab975eccee216ed15"
I20260807 06:38:14.330621 14536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfd801fa29284d2293f8c93932ee7d82" candidate_uuid: "e17590abf34543e7a439cbea449c1bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490edf2139c542a4bd22b5b6a1c81a13"
I20260807 06:38:14.330675 14347 raft_consensus.cc:3037] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:14.330711 14536 raft_consensus.cc:3037] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:14.331316 14536 raft_consensus.cc:2454] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17590abf34543e7a439cbea449c1bbc in term 1.
I20260807 06:38:14.331486 14347 raft_consensus.cc:2454] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17590abf34543e7a439cbea449c1bbc in term 1.
I20260807 06:38:14.331488 14640 leader_election.cc:302] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490edf2139c542a4bd22b5b6a1c81a13, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:14.331595 14839 raft_consensus.cc:2781] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:14.331755 14839 raft_consensus.cc:686] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 1 LEADER]: Becoming Leader. State: Replica: e17590abf34543e7a439cbea449c1bbc, State: Running, Role: LEADER
I20260807 06:38:14.331840 14839 consensus_queue.cc:237] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:14.332616 14128 catalog_manager.cc:5495] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc reported cstate change: term changed from 0 to 1, leader changed from <none> to e17590abf34543e7a439cbea449c1bbc (127.11.114.195). New cstate: current_term: 1 leader_uuid: "e17590abf34543e7a439cbea449c1bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } health_report { overall_health: HEALTHY } } }
I20260807 06:38:14.359557 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.356846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37972 (local address 127.11.114.195:43163)
0807 06:38:14.356902 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.356904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.356912 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:14.356964 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.356965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.356984 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.357039 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:14.357156 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.357739 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.358510 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.358799 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.358890 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.359150 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.359322 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.359389 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.359444 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.359466 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:38:14.365125 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.362434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:48295 (local address 127.11.114.194:43765)
0807 06:38:14.362505 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.362507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.362549 (+ 42us) server_negotiation.cc:407] Connection header received
0807 06:38:14.362614 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.362615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.362624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.362680 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:14.362801 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.363280 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.364115 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.364312 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.364374 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.364851 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.364903 (+ 52us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.364957 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.365036 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.365057 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:38:14.365149 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.362319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43765 (local address 127.11.114.195:48295)
0807 06:38:14.362480 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.362490 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.362560 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.362692 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.362692 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.362694 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:14.362747 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.362748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.363385 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.363386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.364025 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.364027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.364664 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.364665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.364715 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.364727 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.364975 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.364976 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.365011 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.365047 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260807 06:38:14.367396 14536 raft_consensus.cc:1261] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Refusing update from remote peer e17590abf34543e7a439cbea449c1bbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:14.367405 14347 raft_consensus.cc:1261] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Refusing update from remote peer e17590abf34543e7a439cbea449c1bbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:14.367681 14839 consensus_queue.cc:1035] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:14.367847 14917 consensus_queue.cc:1035] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:14.375322 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.372839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:58983 (local address 127.11.114.195:43163)
0807 06:38:14.372873 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:14.372875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:14.372883 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:14.372925 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:14.372926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:14.372936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:14.372989 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:14.373089 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.373628 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.374589 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.374868 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.374951 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.375101 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.375133 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.375181 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:14.375239 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 06:38:14.375259 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:38:14.375344 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:14.372761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:43163 (local address 127.11.114.194:58983)
0807 06:38:14.372817 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:14.372829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:14.372875 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:14.372992 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:14.372992 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:14.372993 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:14.373034 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.373035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.373721 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.373723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.374484 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:14.374487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:14.374949 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:14.374951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:14.375006 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:14.375012 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:14.375181 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:14.375183 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:38:14.375234 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:38:14.375276 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 06:38:14.375666 14871 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 06:38:15.897687 14127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:15.897840 14127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:15.898824 14127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: Sending DeleteTablet for 3 replicas of tablet bfd801fa29284d2293f8c93932ee7d82
I20260807 06:38:15.899158 14505 tablet_service.cc:1514] Processing DeleteTablet for tablet bfd801fa29284d2293f8c93932ee7d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:15 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:50764
I20260807 06:38:15.899385 14320 tablet_service.cc:1514] Processing DeleteTablet for tablet bfd801fa29284d2293f8c93932ee7d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:15 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:48374
I20260807 06:38:15.899399 14962 tablet_replica.cc:330] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: stopping tablet replica
I20260807 06:38:15.899474 14962 raft_consensus.cc:2229] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:15.899533 14962 raft_consensus.cc:2258] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:15.899566 14710 tablet_service.cc:1514] Processing DeleteTablet for tablet bfd801fa29284d2293f8c93932ee7d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:15 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260807 06:38:15.899760 14964 tablet_replica.cc:330] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: stopping tablet replica
I20260807 06:38:15.899843 14964 raft_consensus.cc:2229] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:15.899950 14964 raft_consensus.cc:2258] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:15.900249 14964 ts_tablet_manager.cc:1905] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:15.901281 14962 ts_tablet_manager.cc:1905] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:15.901674 14963 tablet_replica.cc:330] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: stopping tablet replica
I20260807 06:38:15.901752 14963 raft_consensus.cc:2229] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:15.901888 14963 raft_consensus.cc:2258] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:15.902192 14963 ts_tablet_manager.cc:1905] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:15.903666 14962 ts_tablet_manager.cc:1918] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:38:15.903744 14962 log.cc:1196] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/bfd801fa29284d2293f8c93932ee7d82
I20260807 06:38:15.904031 14963 ts_tablet_manager.cc:1918] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:38:15.904038 14962 ts_tablet_manager.cc:1939] T bfd801fa29284d2293f8c93932ee7d82 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting consensus metadata
I20260807 06:38:15.904098 14963 log.cc:1196] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/bfd801fa29284d2293f8c93932ee7d82
I20260807 06:38:15.904373 14112 catalog_manager.cc:4841] TS 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765): tablet bfd801fa29284d2293f8c93932ee7d82 (table test-table [id=9914a1da4dab465698e630c63eb87b1f]) successfully deleted
I20260807 06:38:15.904325 14963 ts_tablet_manager.cc:1939] T bfd801fa29284d2293f8c93932ee7d82 P 3e69c432863b44dab975eccee216ed15: Deleting consensus metadata
I20260807 06:38:15.904678 14964 ts_tablet_manager.cc:1918] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:38:15.904773 14964 log.cc:1196] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/bfd801fa29284d2293f8c93932ee7d82
I20260807 06:38:15.904861 14113 catalog_manager.cc:4841] TS 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533): tablet bfd801fa29284d2293f8c93932ee7d82 (table test-table [id=9914a1da4dab465698e630c63eb87b1f]) successfully deleted
I20260807 06:38:15.905068 14964 ts_tablet_manager.cc:1939] T bfd801fa29284d2293f8c93932ee7d82 P e17590abf34543e7a439cbea449c1bbc: Deleting consensus metadata
I20260807 06:38:15.905423 14113 catalog_manager.cc:4841] TS e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163): tablet bfd801fa29284d2293f8c93932ee7d82 (table test-table [id=9914a1da4dab465698e630c63eb87b1f]) successfully deleted
Aug 07 06:38:15 dist-test-slave-gkw7 krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:38:15 dist-test-slave-gkw7 krb5kdc[14060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:15.929581 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:15.918467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45860 (local address 127.11.114.254:35867)
0807 06:38:15.918650 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:15.918652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:15.918663 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:38:15.918756 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:15.918757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:15.918768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:15.918825 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:15.919417 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:15.920063 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:15.920859 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:15.921058 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:15.921142 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:15.923562 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:15.923564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:15.923564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:15.923571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:15.928230 (+ 4659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:15.928598 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:15.928599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:15.928600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:15.928667 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:15.928858 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:15.928858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:15.928859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:15.928991 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:15.929120 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:15.929256 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 06:38:15.929390 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260807 06:38:15.931311 14127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:15.931475 14127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:15.933069 14320 tablet_service.cc:1467] Processing CreateTablet for tablet 381da532de574b1482565661c39bc1a3 (DEFAULT_TABLE table=test-table [id=5f111f8d3c5e415d94de3f5cc96d241e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:15.933085 14505 tablet_service.cc:1467] Processing CreateTablet for tablet 381da532de574b1482565661c39bc1a3 (DEFAULT_TABLE table=test-table [id=5f111f8d3c5e415d94de3f5cc96d241e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:15.933233 14320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381da532de574b1482565661c39bc1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:15.933256 14505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381da532de574b1482565661c39bc1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:15.934870 14973 tablet_bootstrap.cc:492] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: Bootstrap starting.
I20260807 06:38:15.934899 14974 tablet_bootstrap.cc:492] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: Bootstrap starting.
I20260807 06:38:15.934926 14710 tablet_service.cc:1467] Processing CreateTablet for tablet 381da532de574b1482565661c39bc1a3 (DEFAULT_TABLE table=test-table [id=5f111f8d3c5e415d94de3f5cc96d241e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:15.935046 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381da532de574b1482565661c39bc1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:15.935303 14974 tablet_bootstrap.cc:654] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:15.935360 14973 tablet_bootstrap.cc:654] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:15.935988 14973 tablet_bootstrap.cc:492] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: No bootstrap required, opened a new log
I20260807 06:38:15.936048 14973 ts_tablet_manager.cc:1397] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:15.936183 14973 raft_consensus.cc:348] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.936260 14973 raft_consensus.cc:374] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:15.936281 14973 raft_consensus.cc:729] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e69c432863b44dab975eccee216ed15, State: Initialized, Role: FOLLOWER
I20260807 06:38:15.936332 14973 consensus_queue.cc:260] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.936496 14973 ts_tablet_manager.cc:1428] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:15.936631 14979 tablet_bootstrap.cc:492] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: Bootstrap starting.
I20260807 06:38:15.937098 14974 tablet_bootstrap.cc:492] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: No bootstrap required, opened a new log
I20260807 06:38:15.937150 14979 tablet_bootstrap.cc:654] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:15.937158 14974 ts_tablet_manager.cc:1397] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:15.937279 14974 raft_consensus.cc:348] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.937364 14974 raft_consensus.cc:374] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:15.937388 14974 raft_consensus.cc:729] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490edf2139c542a4bd22b5b6a1c81a13, State: Initialized, Role: FOLLOWER
I20260807 06:38:15.937462 14974 consensus_queue.cc:260] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.937629 14974 ts_tablet_manager.cc:1428] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:15.937981 14979 tablet_bootstrap.cc:492] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: No bootstrap required, opened a new log
I20260807 06:38:15.938078 14979 ts_tablet_manager.cc:1397] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:15.938202 14979 raft_consensus.cc:348] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.938283 14979 raft_consensus.cc:374] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:15.938306 14979 raft_consensus.cc:729] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17590abf34543e7a439cbea449c1bbc, State: Initialized, Role: FOLLOWER
I20260807 06:38:15.938364 14979 consensus_queue.cc:260] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.938476 14979 ts_tablet_manager.cc:1428] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:15.958901 14838 raft_consensus.cc:482] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:15.959021 14838 raft_consensus.cc:504] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.959232 14838 leader_election.cc:288] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533), e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163)
I20260807 06:38:15.959431 14742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381da532de574b1482565661c39bc1a3" candidate_uuid: "490edf2139c542a4bd22b5b6a1c81a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17590abf34543e7a439cbea449c1bbc" is_pre_election: true
I20260807 06:38:15.959510 14742 raft_consensus.cc:2454] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 490edf2139c542a4bd22b5b6a1c81a13 in term 0.
I20260807 06:38:15.959444 14347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381da532de574b1482565661c39bc1a3" candidate_uuid: "490edf2139c542a4bd22b5b6a1c81a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e69c432863b44dab975eccee216ed15" is_pre_election: true
I20260807 06:38:15.959595 14347 raft_consensus.cc:2454] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 490edf2139c542a4bd22b5b6a1c81a13 in term 0.
I20260807 06:38:15.959698 14441 leader_election.cc:302] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490edf2139c542a4bd22b5b6a1c81a13, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:15.959810 14838 raft_consensus.cc:2781] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:15.959846 14838 raft_consensus.cc:482] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:15.959874 14838 raft_consensus.cc:3037] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:15.960597 14838 raft_consensus.cc:504] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.960748 14838 leader_election.cc:288] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [CANDIDATE]: Term 1 election: Requested vote from peers 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533), e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163)
I20260807 06:38:15.960914 14347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381da532de574b1482565661c39bc1a3" candidate_uuid: "490edf2139c542a4bd22b5b6a1c81a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e69c432863b44dab975eccee216ed15"
I20260807 06:38:15.960937 14742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381da532de574b1482565661c39bc1a3" candidate_uuid: "490edf2139c542a4bd22b5b6a1c81a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17590abf34543e7a439cbea449c1bbc"
I20260807 06:38:15.961019 14742 raft_consensus.cc:3037] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:15.961030 14347 raft_consensus.cc:3037] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:15.961874 14742 raft_consensus.cc:2454] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 490edf2139c542a4bd22b5b6a1c81a13 in term 1.
I20260807 06:38:15.962041 14347 raft_consensus.cc:2454] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 490edf2139c542a4bd22b5b6a1c81a13 in term 1.
I20260807 06:38:15.962098 14441 leader_election.cc:302] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490edf2139c542a4bd22b5b6a1c81a13, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:15.962203 14838 raft_consensus.cc:2781] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:15.962291 14838 raft_consensus.cc:686] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 LEADER]: Becoming Leader. State: Replica: 490edf2139c542a4bd22b5b6a1c81a13, State: Running, Role: LEADER
I20260807 06:38:15.962350 14838 consensus_queue.cc:237] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:15.963089 14127 catalog_manager.cc:5495] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "490edf2139c542a4bd22b5b6a1c81a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } health_report { overall_health: UNKNOWN } } }
I20260807 06:38:15.975883 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:15.973281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.11.114.194:43765)
0807 06:38:15.973455 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:15.973456 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:15.973462 (+ 6us) server_negotiation.cc:407] Connection header received
0807 06:38:15.973480 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:15.973480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:15.973490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:15.973604 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:15.973767 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:15.974267 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:15.975144 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:15.975359 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:15.975443 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:15.975601 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:15.975636 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:15.975707 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:15.975768 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 06:38:15.975793 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260807 06:38:15.980346 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:15.978220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:55375 (local address 127.11.114.194:43765)
0807 06:38:15.978269 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:15.978271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:15.978279 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:15.978322 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:15.978322 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:15.978332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:15.978383 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:15.978481 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:15.978915 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:15.979704 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:15.979892 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:15.979965 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:15.980082 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:15.980116 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:15.980169 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:15.980250 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 06:38:15.980268 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:38:15.980352 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:15.978089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43765 (local address 127.11.114.194:55375)
0807 06:38:15.978212 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:15.978222 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:15.978270 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:15.978386 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:15.978387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:15.978388 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:15.978429 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:15.978430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:15.978993 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:15.978995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:15.979604 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:15.979606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:15.979962 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:15.979963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:15.980013 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:15.980017 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:15.980185 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:15.980187 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:38:15.980236 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:38:15.980282 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":79,"threads_started":1}
I20260807 06:38:15.982280 14742 raft_consensus.cc:1261] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Refusing update from remote peer 490edf2139c542a4bd22b5b6a1c81a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:15.982299 14347 raft_consensus.cc:1261] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Refusing update from remote peer 490edf2139c542a4bd22b5b6a1c81a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:15.982520 14838 consensus_queue.cc:1035] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:15.982676 14941 consensus_queue.cc:1035] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:15.993757 14131 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:15.993872 14131 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:15.994517 14131 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: Sending DeleteTablet for 3 replicas of tablet 381da532de574b1482565661c39bc1a3
I20260807 06:38:15.994760 14320 tablet_service.cc:1514] Processing DeleteTablet for tablet 381da532de574b1482565661c39bc1a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:15 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:48374
I20260807 06:38:15.994817 14507 tablet_service.cc:1514] Processing DeleteTablet for tablet 381da532de574b1482565661c39bc1a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:15 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:50764
I20260807 06:38:15.994863 14710 tablet_service.cc:1514] Processing DeleteTablet for tablet 381da532de574b1482565661c39bc1a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:15 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260807 06:38:15.994928 14962 tablet_replica.cc:330] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: stopping tablet replica
I20260807 06:38:15.994956 14964 tablet_replica.cc:330] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: stopping tablet replica
I20260807 06:38:15.994957 14963 tablet_replica.cc:330] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: stopping tablet replica
I20260807 06:38:15.994979 14962 raft_consensus.cc:2229] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:15.995014 14964 raft_consensus.cc:2229] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:15.995019 14963 raft_consensus.cc:2229] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:15.995052 14964 raft_consensus.cc:2258] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:15.995059 14963 raft_consensus.cc:2258] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:15.995111 14962 raft_consensus.cc:2258] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:15.995311 14131 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 06:38:15.995402 14964 ts_tablet_manager.cc:1905] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:15.995411 14962 ts_tablet_manager.cc:1905] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 06:38:15.995473 14131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:15.996613 14320 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0be641f3574218bc93c7787e6d97bb (DEFAULT_TABLE table=test-table [id=e45edc3567ee410aa3804bdf3327f724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:15.996632 14507 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0be641f3574218bc93c7787e6d97bb (DEFAULT_TABLE table=test-table [id=e45edc3567ee410aa3804bdf3327f724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:15.996734 14507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0be641f3574218bc93c7787e6d97bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:15.996733 14320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0be641f3574218bc93c7787e6d97bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:15.997118 14963 ts_tablet_manager.cc:1905] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:15.997773 14964 ts_tablet_manager.cc:1918] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:15.997833 14964 log.cc:1196] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/381da532de574b1482565661c39bc1a3
I20260807 06:38:15.998054 14964 ts_tablet_manager.cc:1939] T 381da532de574b1482565661c39bc1a3 P e17590abf34543e7a439cbea449c1bbc: Deleting consensus metadata
I20260807 06:38:15.998386 14962 ts_tablet_manager.cc:1918] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:15.998452 14962 log.cc:1196] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/381da532de574b1482565661c39bc1a3
I20260807 06:38:15.998539 14963 ts_tablet_manager.cc:1918] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:15.998601 14963 log.cc:1196] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/381da532de574b1482565661c39bc1a3
I20260807 06:38:15.998679 14962 ts_tablet_manager.cc:1939] T 381da532de574b1482565661c39bc1a3 P 490edf2139c542a4bd22b5b6a1c81a13: Deleting consensus metadata
I20260807 06:38:15.998827 14963 ts_tablet_manager.cc:1939] T 381da532de574b1482565661c39bc1a3 P 3e69c432863b44dab975eccee216ed15: Deleting consensus metadata
I20260807 06:38:15.998974 14973 tablet_bootstrap.cc:492] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15: Bootstrap starting.
I20260807 06:38:15.999015 14113 catalog_manager.cc:4841] TS e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163): tablet 381da532de574b1482565661c39bc1a3 (table test-table [id=5f111f8d3c5e415d94de3f5cc96d241e]) successfully deleted
I20260807 06:38:15.999055 14974 tablet_bootstrap.cc:492] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: Bootstrap starting.
I20260807 06:38:15.999058 14710 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0be641f3574218bc93c7787e6d97bb (DEFAULT_TABLE table=test-table [id=e45edc3567ee410aa3804bdf3327f724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:15.999159 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0be641f3574218bc93c7787e6d97bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:15.999269 14973 tablet_bootstrap.cc:654] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:15.999431 14974 tablet_bootstrap.cc:654] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:15.999720 14113 catalog_manager.cc:4841] TS 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533): tablet 381da532de574b1482565661c39bc1a3 (table test-table [id=5f111f8d3c5e415d94de3f5cc96d241e]) successfully deleted
I20260807 06:38:15.999897 14112 catalog_manager.cc:4841] TS 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765): tablet 381da532de574b1482565661c39bc1a3 (table test-table [id=5f111f8d3c5e415d94de3f5cc96d241e]) successfully deleted
I20260807 06:38:16.000223 14974 tablet_bootstrap.cc:492] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: No bootstrap required, opened a new log
I20260807 06:38:16.000289 14974 ts_tablet_manager.cc:1397] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:16.000350 14979 tablet_bootstrap.cc:492] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: Bootstrap starting.
I20260807 06:38:16.000398 14974 raft_consensus.cc:348] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.000461 14974 raft_consensus.cc:374] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:16.000485 14974 raft_consensus.cc:729] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490edf2139c542a4bd22b5b6a1c81a13, State: Initialized, Role: FOLLOWER
I20260807 06:38:16.000537 14974 consensus_queue.cc:260] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.000625 14974 ts_tablet_manager.cc:1428] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:16.000705 14979 tablet_bootstrap.cc:654] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:16.001255 14979 tablet_bootstrap.cc:492] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: No bootstrap required, opened a new log
I20260807 06:38:16.001322 14979 ts_tablet_manager.cc:1397] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:16.001435 14979 raft_consensus.cc:348] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.001502 14979 raft_consensus.cc:374] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:16.001545 14979 raft_consensus.cc:729] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17590abf34543e7a439cbea449c1bbc, State: Initialized, Role: FOLLOWER
I20260807 06:38:16.001605 14979 consensus_queue.cc:260] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.001700 14979 ts_tablet_manager.cc:1428] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:16.009168 14973 tablet_bootstrap.cc:492] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15: No bootstrap required, opened a new log
I20260807 06:38:16.009256 14973 ts_tablet_manager.cc:1397] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260807 06:38:16.009385 14973 raft_consensus.cc:348] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.009478 14973 raft_consensus.cc:374] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:16.009506 14973 raft_consensus.cc:729] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e69c432863b44dab975eccee216ed15, State: Initialized, Role: FOLLOWER
I20260807 06:38:16.009578 14973 consensus_queue.cc:260] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.009699 14973 ts_tablet_manager.cc:1428] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:16.212523 14927 raft_consensus.cc:482] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:16.212631 14927 raft_consensus.cc:504] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.212823 14927 leader_election.cc:288] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533), 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765)
I20260807 06:38:16.213085 14536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0be641f3574218bc93c7787e6d97bb" candidate_uuid: "e17590abf34543e7a439cbea449c1bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490edf2139c542a4bd22b5b6a1c81a13" is_pre_election: true
I20260807 06:38:16.213081 14347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0be641f3574218bc93c7787e6d97bb" candidate_uuid: "e17590abf34543e7a439cbea449c1bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e69c432863b44dab975eccee216ed15" is_pre_election: true
I20260807 06:38:16.213171 14536 raft_consensus.cc:2454] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17590abf34543e7a439cbea449c1bbc in term 0.
I20260807 06:38:16.213199 14347 raft_consensus.cc:2454] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17590abf34543e7a439cbea449c1bbc in term 0.
I20260807 06:38:16.213349 14640 leader_election.cc:302] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490edf2139c542a4bd22b5b6a1c81a13, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:16.213439 14927 raft_consensus.cc:2781] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:16.213481 14927 raft_consensus.cc:482] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:16.213511 14927 raft_consensus.cc:3037] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:16.214320 14927 raft_consensus.cc:504] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.214458 14927 leader_election.cc:288] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [CANDIDATE]: Term 1 election: Requested vote from peers 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533), 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765)
I20260807 06:38:16.214587 14347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0be641f3574218bc93c7787e6d97bb" candidate_uuid: "e17590abf34543e7a439cbea449c1bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e69c432863b44dab975eccee216ed15"
I20260807 06:38:16.214654 14347 raft_consensus.cc:3037] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:16.214607 14536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0be641f3574218bc93c7787e6d97bb" candidate_uuid: "e17590abf34543e7a439cbea449c1bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490edf2139c542a4bd22b5b6a1c81a13"
I20260807 06:38:16.214696 14536 raft_consensus.cc:3037] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:16.215308 14347 raft_consensus.cc:2454] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17590abf34543e7a439cbea449c1bbc in term 1.
I20260807 06:38:16.215456 14536 raft_consensus.cc:2454] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17590abf34543e7a439cbea449c1bbc in term 1.
I20260807 06:38:16.215507 14641 leader_election.cc:302] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e69c432863b44dab975eccee216ed15, e17590abf34543e7a439cbea449c1bbc; no voters:
I20260807 06:38:16.215621 14927 raft_consensus.cc:2781] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:16.215679 14927 raft_consensus.cc:686] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 1 LEADER]: Becoming Leader. State: Replica: e17590abf34543e7a439cbea449c1bbc, State: Running, Role: LEADER
I20260807 06:38:16.215723 14927 consensus_queue.cc:237] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } }
I20260807 06:38:16.216382 14131 catalog_manager.cc:5495] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc reported cstate change: term changed from 0 to 1, leader changed from <none> to e17590abf34543e7a439cbea449c1bbc (127.11.114.195). New cstate: current_term: 1 leader_uuid: "e17590abf34543e7a439cbea449c1bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17590abf34543e7a439cbea449c1bbc" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 43163 } health_report { overall_health: HEALTHY } } }
I20260807 06:38:16.229569 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:16.227381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53940 (local address 127.11.114.195:43163)
0807 06:38:16.227523 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:16.227525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:16.227531 (+ 6us) server_negotiation.cc:407] Connection header received
0807 06:38:16.227548 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:16.227548 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:16.227557 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:16.227610 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:16.227737 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:16.228236 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:16.228957 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:16.229166 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:16.229221 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:16.229322 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:16.229347 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:16.229381 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:16.229430 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 06:38:16.229446 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260807 06:38:16.231707 14536 raft_consensus.cc:1261] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Refusing update from remote peer e17590abf34543e7a439cbea449c1bbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:16.231877 14347 raft_consensus.cc:1261] T 5b0be641f3574218bc93c7787e6d97bb P 3e69c432863b44dab975eccee216ed15 [term 1 FOLLOWER]: Refusing update from remote peer e17590abf34543e7a439cbea449c1bbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:16.231978 14953 consensus_queue.cc:1035] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "490edf2139c542a4bd22b5b6a1c81a13" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:16.232086 14927 consensus_queue.cc:1035] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e69c432863b44dab975eccee216ed15" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:38:16.236698 14935 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 06:38:17.760053 14127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:17.760170 14127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:17.761269 14127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a8fca9de83946ceb9d7093b42c78e71: Sending DeleteTablet for 3 replicas of tablet 5b0be641f3574218bc93c7787e6d97bb
I20260807 06:38:17.761637 14320 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0be641f3574218bc93c7787e6d97bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:17 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:48374
I20260807 06:38:17.761631 14507 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0be641f3574218bc93c7787e6d97bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:17 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:50764
I20260807 06:38:17.761637 14710 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0be641f3574218bc93c7787e6d97bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:17 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260807 06:38:17.761763 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 14234
I20260807 06:38:17.770411 15049 tablet_replica.cc:330] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: stopping tablet replica
I20260807 06:38:17.770530 15049 raft_consensus.cc:2229] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:17.770663 15049 raft_consensus.cc:2258] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:17.770713 15046 tablet_replica.cc:330] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: stopping tablet replica
I20260807 06:38:17.770785 15046 raft_consensus.cc:2229] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:17.770864 15046 raft_consensus.cc:2258] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:17.771121 15046 ts_tablet_manager.cc:1905] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:17.771150 15049 ts_tablet_manager.cc:1905] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:17.772950 15046 ts_tablet_manager.cc:1918] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:38:17.772950 15049 ts_tablet_manager.cc:1918] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:38:17.773017 15049 log.cc:1196] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/5b0be641f3574218bc93c7787e6d97bb
I20260807 06:38:17.773017 15046 log.cc:1196] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/5b0be641f3574218bc93c7787e6d97bb
I20260807 06:38:17.773258 15046 ts_tablet_manager.cc:1939] T 5b0be641f3574218bc93c7787e6d97bb P 490edf2139c542a4bd22b5b6a1c81a13: Deleting consensus metadata
I20260807 06:38:17.773285 15049 ts_tablet_manager.cc:1939] T 5b0be641f3574218bc93c7787e6d97bb P e17590abf34543e7a439cbea449c1bbc: Deleting consensus metadata
I20260807 06:38:17.773666 14112 catalog_manager.cc:4841] TS 490edf2139c542a4bd22b5b6a1c81a13 (127.11.114.194:43765): tablet 5b0be641f3574218bc93c7787e6d97bb (table test-table [id=e45edc3567ee410aa3804bdf3327f724]) successfully deleted
I20260807 06:38:17.773736 14113 catalog_manager.cc:4841] TS e17590abf34543e7a439cbea449c1bbc (127.11.114.195:43163): tablet 5b0be641f3574218bc93c7787e6d97bb (table test-table [id=e45edc3567ee410aa3804bdf3327f724]) successfully deleted
W20260807 06:38:17.776410 14113 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:37533 (error 108)
I20260807 06:38:17.776600 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 14416
I20260807 06:38:17.780992 15053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:17.780746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:37533 (local address 127.0.0.1:44170)
0807 06:38:17.780891 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:17.780953 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
W20260807 06:38:17.781126 14113 catalog_manager.cc:4584] TS 3e69c432863b44dab975eccee216ed15 (127.11.114.193:37533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b0be641f3574218bc93c7787e6d97bb: Network error: Client connection negotiation failed: client connection to 127.11.114.193:37533: connect: Connection refused (error 111)
I20260807 06:38:17.787401 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 14617
I20260807 06:38:17.798637 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 14084
2026-08-07T06:38:17Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6265 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:17 dist-test-slave-gkw7 krb5kdc[15057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:17 dist-test-slave-gkw7 krb5kdc[15057](info): set up 2 sockets
Aug 07 06:38:17 dist-test-slave-gkw7 krb5kdc[15057](info): commencing operation
krb5kdc: starting...
W20260807 06:38:19.090593 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:19.216305 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40181
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:40181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:19.301136 15074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:19.301337 15074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:19.301358 15074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:19.302539 15074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:19.302582 15074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:19.302597 15074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:19.302610 15074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:19.302623 15074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:19.304051 15074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:40181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:19.304255 15074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:19.304418 15074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:19.309312 15080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:19.309340 15085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:19.309327 15079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:19.309381 15074 server_base.cc:1034] running on GCE node
I20260807 06:38:19.309820 15074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:19.310109 15074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:19.311306 15074 hybrid_clock.cc:648] HybridClock initialized: now 1786084699311298 us; error 36 us; skew 500 ppm
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:19.314066 15074 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:19.314465 15074 webserver.cc:460] Webserver started at http://127.11.114.254:42235/ using document root <none> and password file <none>
I20260807 06:38:19.314666 15074 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:19.314706 15074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:19.314792 15074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:19.315855 15074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "0a2b4f87c1404b54bae44409c60aac26"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "8a697a659e96cad335a971a37c8ca493"
server_key_iv: "1591fdc8f3b009ac3d367433a62d0a22"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.316176 15074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "0a2b4f87c1404b54bae44409c60aac26"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "8a697a659e96cad335a971a37c8ca493"
server_key_iv: "1591fdc8f3b009ac3d367433a62d0a22"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.317474 15074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:38:19.318213 15094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.318393 15074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:19.318506 15074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "0a2b4f87c1404b54bae44409c60aac26"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "8a697a659e96cad335a971a37c8ca493"
server_key_iv: "1591fdc8f3b009ac3d367433a62d0a22"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.318588 15074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:19.329660 15074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:19.331910 15074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:19.332084 15074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:19.335992 15074 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:40181
I20260807 06:38:19.336031 15164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:40181 every 8 connection(s)
I20260807 06:38:19.336382 15074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:19.337250 15165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:19.339493 15165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26: Bootstrap starting.
I20260807 06:38:19.340142 15165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:19.340425 15165 log.cc:824] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:19.341202 15165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26: No bootstrap required, opened a new log
I20260807 06:38:19.341450 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 15074
I20260807 06:38:19.341565 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:19.341765 11723 external_mini_cluster.cc:1428] Setting key a043504fb4bce0f91f835b8956a68eb9
I20260807 06:38:19.343456 15165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2b4f87c1404b54bae44409c60aac26" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40181 } }
I20260807 06:38:19.343591 15165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:19.343614 15165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2b4f87c1404b54bae44409c60aac26, State: Initialized, Role: FOLLOWER
I20260807 06:38:19.343748 15165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2b4f87c1404b54bae44409c60aac26" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40181 } }
I20260807 06:38:19.343822 15165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:19.343843 15165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:19.343911 15165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:19.344666 15165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2b4f87c1404b54bae44409c60aac26" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40181 } }
I20260807 06:38:19.344780 15165 leader_election.cc:302] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a2b4f87c1404b54bae44409c60aac26; no voters:
I20260807 06:38:19.344983 15165 leader_election.cc:288] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:19.345012 15174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:19.345127 15174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [term 1 LEADER]: Becoming Leader. State: Replica: 0a2b4f87c1404b54bae44409c60aac26, State: Running, Role: LEADER
I20260807 06:38:19.345213 15174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2b4f87c1404b54bae44409c60aac26" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40181 } }
I20260807 06:38:19.345280 15165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:19.345517 15175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a2b4f87c1404b54bae44409c60aac26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2b4f87c1404b54bae44409c60aac26" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40181 } } }
I20260807 06:38:19.345783 15175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:19.345506 15174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a2b4f87c1404b54bae44409c60aac26. Latest consensus state: current_term: 1 leader_uuid: "0a2b4f87c1404b54bae44409c60aac26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2b4f87c1404b54bae44409c60aac26" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40181 } } }
I20260807 06:38:19.346031 15174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:19.346341 15184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:19.346987 15184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:19.348687 15184 catalog_manager.cc:1347] Generated new cluster ID: b9debcb9d5ec47e5b314117dd2e30e19
I20260807 06:38:19.348742 15184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:19.355721 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:19.342319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.11.114.254:40181)
0807 06:38:19.342505 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:19.342513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:19.342538 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:38:19.343646 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:19.343658 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:19.343745 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:19.343915 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:19.344802 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.346255 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.347342 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.347942 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.348071 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:19.351888 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:19.351892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:19.351893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:19.351904 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:19.354063 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:19.354463 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:19.354467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:19.354467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:19.354593 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:19.354805 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:19.354806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:19.354806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:19.355132 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:19.355294 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:19.355377 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 06:38:19.355521 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260807 06:38:19.371284 15184 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:19.371850 15184 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:19.377172 15184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a2b4f87c1404b54bae44409c60aac26: Generated new TSK 0
I20260807 06:38:19.377367 15184 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:19.412576 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40181
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:19.502110 15199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:19.502327 15199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:19.502351 15199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:19.503556 15199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:19.503608 15199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:19.503636 15199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:19.505162 15199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:19.505412 15199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:19.505714 15199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:38:19.510895 15199 server_base.cc:1034] running on GCE node
W20260807 06:38:19.510874 15209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:19.510864 15205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:19.510864 15211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:19.511328 15199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:19.511637 15199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:19.512789 15199 hybrid_clock.cc:648] HybridClock initialized: now 1786084699512771 us; error 41 us; skew 500 ppm
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:19.515441 15199 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:19.515969 15199 webserver.cc:460] Webserver started at http://127.11.114.193:37045/ using document root <none> and password file <none>
I20260807 06:38:19.516234 15199 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:19.516309 15199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:19.516456 15199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:19.517511 15199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "3f78cb51e7c8405395feb2bcb8307873"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "511380fec4907a6e86a6a970010a8792"
server_key_iv: "acecb70f7d8545de9d9a71f550ac512b"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.517935 15199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "3f78cb51e7c8405395feb2bcb8307873"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "511380fec4907a6e86a6a970010a8792"
server_key_iv: "acecb70f7d8545de9d9a71f550ac512b"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.519302 15199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:19.520045 15222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.520274 15199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:19.520372 15199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "3f78cb51e7c8405395feb2bcb8307873"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "511380fec4907a6e86a6a970010a8792"
server_key_iv: "acecb70f7d8545de9d9a71f550ac512b"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.520453 15199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:19.532514 15199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:19.534482 15199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:19.534643 15199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:19.534868 15199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:19.535172 15199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:19.535224 15199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.535257 15199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:19.535283 15199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.541210 15199 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:42559
I20260807 06:38:19.541265 15380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:42559 every 8 connection(s)
I20260807 06:38:19.541587 15199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:19.548734 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 15199
I20260807 06:38:19.548832 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:19.549041 11723 external_mini_cluster.cc:1428] Setting key 7b39aad4eeba5044ac8c835a2b20adb8
I20260807 06:38:19.550989 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:19.542293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:40619 (local address 127.11.114.254:40181)
0807 06:38:19.542376 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:19.542379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:19.542558 (+ 179us) server_negotiation.cc:407] Connection header received
0807 06:38:19.543000 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:19.543001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:19.543011 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:19.543063 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:19.543969 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.544498 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.545585 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.545789 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.545844 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:19.548042 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:19.548044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:19.548044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:19.548050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:19.549449 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:19.549945 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:19.549946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:19.549947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:19.550012 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:19.550323 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:19.550324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:19.550324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:19.550444 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:19.550519 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:19.550798 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 06:38:19.550889 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 06:38:19.551169 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:19.542255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40181 (local address 127.11.114.193:40619)
0807 06:38:19.542442 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:19.542477 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:19.542922 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:19.543128 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:19.543131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:19.543703 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:19.543890 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:19.543894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.544599 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.544600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:19.545478 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:19.545481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.545867 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.545868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:19.546050 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:19.546532 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:19.546540 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:19.547962 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:19.549520 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:19.549545 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:19.549548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:19.549855 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:19.550101 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:19.550102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:19.550103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:19.550248 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:19.550520 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:19.550528 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:19.550670 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 06:38:19.550777 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 06:38:19.550931 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260807 06:38:19.551940 15381 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40181
I20260807 06:38:19.552022 15381 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:19.552229 15381 heartbeater.cc:511] Master 127.11.114.254:40181 requested a full tablet report, sending...
I20260807 06:38:19.552839 15115 ts_manager.cc:194] Registered new tserver with Master: 3f78cb51e7c8405395feb2bcb8307873 (127.11.114.193:42559)
I20260807 06:38:19.553809 15115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:40619
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:19.576032 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40181
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:19.660297 15389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:19.660584 15389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:19.660610 15389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:19.661732 15389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:19.661765 15389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:19.661809 15389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:19.663159 15389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:19.663334 15389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:19.663506 15389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:19.668553 15396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:19.668533 15395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:19.668538 15398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:19.668695 15389 server_base.cc:1034] running on GCE node
I20260807 06:38:19.669042 15389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:19.669350 15389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:19.670526 15389 hybrid_clock.cc:648] HybridClock initialized: now 1786084699670476 us; error 84 us; skew 500 ppm
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:19.673164 15389 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:19.673741 15389 webserver.cc:460] Webserver started at http://127.11.114.194:43789/ using document root <none> and password file <none>
I20260807 06:38:19.673960 15389 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:19.674003 15389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:19.674142 15389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:19.675207 15389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "0dc53f120027455e8784feefdfbf00dc"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "dec415f85306a7d46090c8f39a21ae16"
server_key_iv: "2bfd74b9531ff1ab4a502c8229b5289e"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.675613 15389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "0dc53f120027455e8784feefdfbf00dc"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "dec415f85306a7d46090c8f39a21ae16"
server_key_iv: "2bfd74b9531ff1ab4a502c8229b5289e"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.677153 15389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:19.678005 15412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.678221 15389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:38:19.678321 15389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "0dc53f120027455e8784feefdfbf00dc"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "dec415f85306a7d46090c8f39a21ae16"
server_key_iv: "2bfd74b9531ff1ab4a502c8229b5289e"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.678376 15389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:19.691558 15389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:19.693907 15389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:19.694017 15389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:19.694192 15389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:19.694514 15389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:19.694550 15389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.694595 15389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:19.694622 15389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.700490 15389 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:43795
I20260807 06:38:19.700570 15559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:43795 every 8 connection(s)
I20260807 06:38:19.700881 15389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:19.701072 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 15389
I20260807 06:38:19.701164 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:19.701372 11723 external_mini_cluster.cc:1428] Setting key f4ee3fd2792c8dfe4abae2d9b00b843c
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
I20260807 06:38:19.713043 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:19.701684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:58789 (local address 127.11.114.254:40181)
0807 06:38:19.701752 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:19.701754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:19.702098 (+ 344us) server_negotiation.cc:407] Connection header received
0807 06:38:19.702861 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:19.702862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:19.702872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:19.702922 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:19.704503 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.705302 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.706989 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.707198 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.707309 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:19.710037 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:19.710039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:19.710039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:19.710047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:19.711555 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:19.711997 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:19.711998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:19.711999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:19.712063 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:19.712373 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:19.712374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:19.712374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:19.712494 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:19.712572 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:19.712864 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 06:38:19.712948 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:38:19.713296 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:19.701764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40181 (local address 127.11.114.194:58789)
0807 06:38:19.701958 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:19.702013 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:19.702750 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:19.702999 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:19.703003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:19.704114 (+ 1111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:19.704390 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:19.704397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.705421 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.705422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:19.706868 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:19.706872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.707319 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.707320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:19.707462 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:19.708153 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:19.708163 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:19.709934 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:19.711674 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:19.711682 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:19.711684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:19.711916 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:19.712165 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:19.712166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:19.712166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:19.712307 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:19.712577 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:19.712584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:19.712727 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 06:38:19.712840 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 06:38:19.712999 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
I20260807 06:38:19.714013 15560 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40181
I20260807 06:38:19.714208 15560 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:19.714470 15560 heartbeater.cc:511] Master 127.11.114.254:40181 requested a full tablet report, sending...
I20260807 06:38:19.714926 15115 ts_manager.cc:194] Registered new tserver with Master: 0dc53f120027455e8784feefdfbf00dc (127.11.114.194:43795)
I20260807 06:38:19.715314 15115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:58789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:19.730448 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40181
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:19.812551 15566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:19.812770 15566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:19.812798 15566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:19.813998 15566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:19.814040 15566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:19.814067 15566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:19.815593 15566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:19.815872 15566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:19.816105 15566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:19.821272 15572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:19.821328 15571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:19.821486 15566 server_base.cc:1034] running on GCE node
W20260807 06:38:19.821557 15574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:19.821799 15566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:19.822053 15566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:19.823223 15566 hybrid_clock.cc:648] HybridClock initialized: now 1786084699823212 us; error 28 us; skew 500 ppm
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:19.826016 15566 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:19.826498 15566 webserver.cc:460] Webserver started at http://127.11.114.195:42929/ using document root <none> and password file <none>
I20260807 06:38:19.826694 15566 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:19.826735 15566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:19.826826 15566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:19.827818 15566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "dd92460352a849a0a331f5f0db834251"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "af09839b6d413a9660a2e0d3e7501cb2"
server_key_iv: "12ae4dd5a708be2ea24de0a14867a3d4"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.828130 15566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "dd92460352a849a0a331f5f0db834251"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "af09839b6d413a9660a2e0d3e7501cb2"
server_key_iv: "12ae4dd5a708be2ea24de0a14867a3d4"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.829413 15566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:38:19.830308 15584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.830505 15566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:19.830608 15566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "dd92460352a849a0a331f5f0db834251"
format_stamp: "Formatted at 2026-08-07 06:38:19 on dist-test-slave-gkw7"
server_key: "af09839b6d413a9660a2e0d3e7501cb2"
server_key_iv: "12ae4dd5a708be2ea24de0a14867a3d4"
server_key_version: "encryptionkey@0"
I20260807 06:38:19.830679 15566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:19.846392 15566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:19.848477 15566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:19.848636 15566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:19.848865 15566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:19.849188 15566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:19.849222 15566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.849246 15566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:19.849289 15566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:19.855067 15566 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:37385
I20260807 06:38:19.855142 15729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:37385 every 8 connection(s)
I20260807 06:38:19.855464 15566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084699, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:19.864815 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:19.856223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:55273 (local address 127.11.114.254:40181)
0807 06:38:19.856308 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:19.856310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:19.856490 (+ 180us) server_negotiation.cc:407] Connection header received
0807 06:38:19.856949 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:19.856950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:19.856960 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:19.857013 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:19.858020 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.858539 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.859551 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.859742 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.859798 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:19.861951 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:19.861953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:19.861954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:19.861961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:19.863373 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:19.863803 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:19.863804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:19.863804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:19.863867 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:19.864138 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:19.864139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:19.864139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:19.864258 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:19.864336 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:19.864611 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 06:38:19.864692 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:38:19.865010 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:19.856182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40181 (local address 127.11.114.195:55273)
0807 06:38:19.856370 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:19.856407 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:19.856868 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:19.857074 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:19.857077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:19.857730 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:19.857930 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:19.857935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.858631 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.858632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:19.859474 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:19.859476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:19.859818 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:19.859819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:19.859917 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:19.860375 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:19.860385 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:19.861850 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:19.863478 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:19.863484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:19.863487 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:19.863722 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:19.863943 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:19.863944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:19.863944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:19.864055 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:19.864328 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:19.864334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:19.864476 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 06:38:19.864589 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 06:38:19.864761 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260807 06:38:19.865572 15730 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40181
I20260807 06:38:19.865625 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 15566
I20260807 06:38:19.865684 15730 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:19.865711 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:19.865898 15730 heartbeater.cc:511] Master 127.11.114.254:40181 requested a full tablet report, sending...
I20260807 06:38:19.865929 11723 external_mini_cluster.cc:1428] Setting key 8523a9b1476b10bc4a88caf9cd7a3698
I20260807 06:38:19.866381 15114 ts_manager.cc:194] Registered new tserver with Master: dd92460352a849a0a331f5f0db834251 (127.11.114.195:37385)
I20260807 06:38:19.866825 15114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:55273
I20260807 06:38:19.867740 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:38:19.882675 15173 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:19.881613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.11.114.254:40181)
0807 06:38:19.881669 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:19.881671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:19.881801 (+ 130us) server_negotiation.cc:407] Connection header received
0807 06:38:19.881858 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:19.881859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:19.881868 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:19.881934 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:19.882562 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42250: BlockingRecv error: recv got EOF from 127.0.0.1:42250 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:38:19.883577 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 15199
I20260807 06:38:19.894407 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 15389
I20260807 06:38:19.902837 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 15566
I20260807 06:38:19.908810 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 15074
2026-08-07T06:38:19Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2104 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15748](info): set up 2 sockets
Aug 07 06:38:19 dist-test-slave-gkw7 krb5kdc[15748](info): commencing operation
krb5kdc: starting...
W20260807 06:38:21.199960 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:21 dist-test-slave-gkw7 krb5kdc[15748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:21.324810 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:37837
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:37837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:21.405628 15767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:21.405825 15767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:21.405848 15767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:21.406937 15767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:21.406972 15767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:21.406985 15767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:21.406998 15767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:21.407011 15767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:21.408433 15767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:37837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:37837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:21.408644 15767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:21.408800 15767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:21.413724 15777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:21.413743 15774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:21.413724 15773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:21.414002 15767 server_base.cc:1034] running on GCE node
I20260807 06:38:21.414230 15767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:21.414556 15767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:21.415793 15767 hybrid_clock.cc:648] HybridClock initialized: now 1786084701415756 us; error 53 us; skew 500 ppm
Aug 07 06:38:21 dist-test-slave-gkw7 krb5kdc[15748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:21.421916 15767 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:21.422355 15767 webserver.cc:460] Webserver started at http://127.11.114.254:38077/ using document root <none> and password file <none>
I20260807 06:38:21.422614 15767 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:21.422662 15767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:21.422807 15767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:21.423846 15767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "f3030371bfee443dbedc3a7f24e7b5f0"
format_stamp: "Formatted at 2026-08-07 06:38:21 on dist-test-slave-gkw7"
server_key: "6fe95f4cd77d2cfba8bfd9626667f227"
server_key_iv: "b4dbf99b9f60885f46255982f745731e"
server_key_version: "encryptionkey@0"
I20260807 06:38:21.424216 15767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "f3030371bfee443dbedc3a7f24e7b5f0"
format_stamp: "Formatted at 2026-08-07 06:38:21 on dist-test-slave-gkw7"
server_key: "6fe95f4cd77d2cfba8bfd9626667f227"
server_key_iv: "b4dbf99b9f60885f46255982f745731e"
server_key_version: "encryptionkey@0"
I20260807 06:38:21.425691 15767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:21.426494 15784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:21.426703 15767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:21.426810 15767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "f3030371bfee443dbedc3a7f24e7b5f0"
format_stamp: "Formatted at 2026-08-07 06:38:21 on dist-test-slave-gkw7"
server_key: "6fe95f4cd77d2cfba8bfd9626667f227"
server_key_iv: "b4dbf99b9f60885f46255982f745731e"
server_key_version: "encryptionkey@0"
I20260807 06:38:21.426882 15767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:21.438663 15767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:21.440892 15767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:21.441051 15767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:21.445102 15767 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:37837
I20260807 06:38:21.445150 15855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:37837 every 8 connection(s)
I20260807 06:38:21.445766 15767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:21.446468 15856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:21.448700 15856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0: Bootstrap starting.
I20260807 06:38:21.449307 15856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:21.449574 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 15767
I20260807 06:38:21.449678 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:21.449708 15856 log.cc:824] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:21.449875 11723 external_mini_cluster.cc:1428] Setting key 45c37566fd5706d18295f3484c4dd80d
I20260807 06:38:21.450609 15856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0: No bootstrap required, opened a new log
I20260807 06:38:21.453245 15856 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3030371bfee443dbedc3a7f24e7b5f0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37837 } }
I20260807 06:38:21.453413 15856 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:21.453447 15856 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3030371bfee443dbedc3a7f24e7b5f0, State: Initialized, Role: FOLLOWER
I20260807 06:38:21.453608 15856 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3030371bfee443dbedc3a7f24e7b5f0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37837 } }
I20260807 06:38:21.453685 15856 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:21.453714 15856 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:21.453758 15856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:21.454350 15856 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3030371bfee443dbedc3a7f24e7b5f0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37837 } }
I20260807 06:38:21.454437 15856 leader_election.cc:302] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3030371bfee443dbedc3a7f24e7b5f0; no voters:
I20260807 06:38:21.454658 15856 leader_election.cc:288] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:21.454697 15864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:21.454896 15864 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [term 1 LEADER]: Becoming Leader. State: Replica: f3030371bfee443dbedc3a7f24e7b5f0, State: Running, Role: LEADER
I20260807 06:38:21.454938 15856 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:21.455009 15864 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3030371bfee443dbedc3a7f24e7b5f0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37837 } }
I20260807 06:38:21.456535 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3030371bfee443dbedc3a7f24e7b5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3030371bfee443dbedc3a7f24e7b5f0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37837 } } }
I20260807 06:38:21.456641 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [sys.catalog]: This master's current role is: LEADER
Aug 07 06:38:21 dist-test-slave-gkw7 krb5kdc[15748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
W20260807 06:38:21.456913 15886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 06:38:21.456916 15866 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3030371bfee443dbedc3a7f24e7b5f0. Latest consensus state: current_term: 1 leader_uuid: "f3030371bfee443dbedc3a7f24e7b5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3030371bfee443dbedc3a7f24e7b5f0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37837 } } }
I20260807 06:38:21.457494 15866 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0 [sys.catalog]: This master's current role is: LEADER
W20260807 06:38:21.457005 15886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:21.457679 15888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:21.457882 15888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:21.459496 15888 catalog_manager.cc:1347] Generated new cluster ID: d526a51f602a4349928a61abb88c8db2
I20260807 06:38:21.459550 15888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:21.461242 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:21.450370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.11.114.254:37837)
0807 06:38:21.450548 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:21.450556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:21.450582 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:38:21.451025 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:21.451032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:21.451078 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:21.451188 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:21.452068 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:21.452930 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:21.454019 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:21.454449 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:21.454561 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:21.457480 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:21.457484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:21.457485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:21.457497 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:21.459655 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:21.460063 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:21.460066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:21.460067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:21.460171 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:21.460388 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:21.460389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:21.460390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:21.460665 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:21.460801 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:21.460919 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 06:38:21.461040 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260807 06:38:21.462109 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:38:21.472415 15888 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:21.473188 15888 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:21.481837 15888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3030371bfee443dbedc3a7f24e7b5f0: Generated new TSK 0
I20260807 06:38:21.482009 15888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:38:21.488358 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:21.484606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.11.114.254:37837)
0807 06:38:21.484652 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:21.484655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:21.484771 (+ 116us) server_negotiation.cc:407] Connection header received
0807 06:38:21.484839 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:21.484840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:21.484849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:21.484899 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:21.485331 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:21.485903 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:21.486963 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:21.487163 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:21.487217 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:21.487886 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:21.487887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:21.487887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:21.488063 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:21.488079 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:21.488085 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:21.488131 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:21.488218 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 06:38:21.488241 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:21.490921 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 15767
2026-08-07T06:38:21Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1579 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:21 dist-test-slave-gkw7 krb5kdc[15901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:21 dist-test-slave-gkw7 krb5kdc[15901](info): set up 2 sockets
Aug 07 06:38:21 dist-test-slave-gkw7 krb5kdc[15901](info): commencing operation
krb5kdc: starting...
W20260807 06:38:22.752854 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:22 dist-test-slave-gkw7 krb5kdc[15901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:22.882138 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:36757
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:36757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:22.964798 15919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:22.965049 15919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:22.965086 15919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:22.966260 15919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:22.966336 15919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:22.966360 15919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:22.966382 15919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:22.966413 15919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:22.967936 15919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:36757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:36757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:22.968216 15919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:22.968448 15919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:22.973405 15926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:22.973383 15929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:22.973383 15925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:22.973433 15919 server_base.cc:1034] running on GCE node
I20260807 06:38:22.973901 15919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:22.974241 15919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:22.975387 15919 hybrid_clock.cc:648] HybridClock initialized: now 1786084702975369 us; error 45 us; skew 500 ppm
Aug 07 06:38:22 dist-test-slave-gkw7 krb5kdc[15901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084702, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:22.977941 15919 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:22.978438 15919 webserver.cc:460] Webserver started at http://127.11.114.254:39053/ using document root <none> and password file <none>
I20260807 06:38:22.978688 15919 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:22.978760 15919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:22.978924 15919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:22.979976 15919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "6536a0176a774646bc660d9253c291a1"
format_stamp: "Formatted at 2026-08-07 06:38:22 on dist-test-slave-gkw7"
server_key: "796dd564b20130163401f07f5f1296fd"
server_key_iv: "41702132245d373ade3b2bfbb774b778"
server_key_version: "encryptionkey@0"
I20260807 06:38:22.980365 15919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "6536a0176a774646bc660d9253c291a1"
format_stamp: "Formatted at 2026-08-07 06:38:22 on dist-test-slave-gkw7"
server_key: "796dd564b20130163401f07f5f1296fd"
server_key_iv: "41702132245d373ade3b2bfbb774b778"
server_key_version: "encryptionkey@0"
I20260807 06:38:22.981985 15919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:22.982818 15943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:22.983044 15919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:22.983155 15919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "6536a0176a774646bc660d9253c291a1"
format_stamp: "Formatted at 2026-08-07 06:38:22 on dist-test-slave-gkw7"
server_key: "796dd564b20130163401f07f5f1296fd"
server_key_iv: "41702132245d373ade3b2bfbb774b778"
server_key_version: "encryptionkey@0"
I20260807 06:38:22.983230 15919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:23.002220 15919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:23.004271 15919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:23.004436 15919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:23.008848 15919 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:36757
I20260807 06:38:23.008889 16019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:36757 every 8 connection(s)
I20260807 06:38:23.009294 15919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:23.010488 16020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:23.012862 16020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1: Bootstrap starting.
I20260807 06:38:23.013418 16020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:23.013793 16020 log.cc:824] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:23.014539 16020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1: No bootstrap required, opened a new log
I20260807 06:38:23.016463 16020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6536a0176a774646bc660d9253c291a1" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36757 } }
I20260807 06:38:23.016615 16020 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:23.016639 16020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6536a0176a774646bc660d9253c291a1, State: Initialized, Role: FOLLOWER
I20260807 06:38:23.016779 16020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6536a0176a774646bc660d9253c291a1" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36757 } }
I20260807 06:38:23.016866 16020 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:23.016889 16020 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:23.016918 16020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:23.017601 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 15919
I20260807 06:38:23.017710 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:23.017897 11723 external_mini_cluster.cc:1428] Setting key 5347ff4e982b1a3c1e2bda557538bcd7
I20260807 06:38:23.017936 16020 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6536a0176a774646bc660d9253c291a1" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36757 } }
I20260807 06:38:23.018049 16020 leader_election.cc:302] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6536a0176a774646bc660d9253c291a1; no voters:
I20260807 06:38:23.018241 16020 leader_election.cc:288] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:23.018301 16023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:23.018477 16023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [term 1 LEADER]: Becoming Leader. State: Replica: 6536a0176a774646bc660d9253c291a1, State: Running, Role: LEADER
I20260807 06:38:23.018540 16020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:23.018615 16023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6536a0176a774646bc660d9253c291a1" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36757 } }
I20260807 06:38:23.019061 16023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6536a0176a774646bc660d9253c291a1. Latest consensus state: current_term: 1 leader_uuid: "6536a0176a774646bc660d9253c291a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6536a0176a774646bc660d9253c291a1" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36757 } } }
I20260807 06:38:23.019179 16025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6536a0176a774646bc660d9253c291a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6536a0176a774646bc660d9253c291a1" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36757 } } }
I20260807 06:38:23.019343 16025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:23.019194 16023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:23.019665 16037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:23.020514 16037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:23.022092 16037 catalog_manager.cc:1347] Generated new cluster ID: caf6272211df47c49b649cdaac1f8376
I20260807 06:38:23.022145 16037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:23.031081 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:23.018668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35242 (local address 127.11.114.254:36757)
0807 06:38:23.019267 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:23.019277 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:23.019317 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:38:23.019416 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:23.019421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:23.019466 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:23.020559 (+ 1093us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:23.020679 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.022305 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.023419 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.024099 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.024240 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:23.027224 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:23.027228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:23.027229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:23.027241 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:23.029318 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.029784 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.029787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.029787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.029909 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.030157 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.030158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.030158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.030448 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:23.030593 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:23.030742 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 06:38:23.030873 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":86,"threads_started":1}
I20260807 06:38:23.037319 16037 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:23.037900 16037 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:23.041522 16037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6536a0176a774646bc660d9253c291a1: Generated new TSK 0
I20260807 06:38:23.041749 16037 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:23.072399 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36757
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:23.156090 16050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:23.156291 16050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:23.156312 16050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:23.157518 16050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:23.157589 16050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:23.157634 16050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:23.159118 16050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:23.159353 16050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:23.159603 16050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:23.164746 16057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:23.164742 16059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:23.164803 16050 server_base.cc:1034] running on GCE node
W20260807 06:38:23.164741 16056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:23.165370 16050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:23.165658 16050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:23.166812 16050 hybrid_clock.cc:648] HybridClock initialized: now 1786084703166799 us; error 30 us; skew 500 ppm
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084703, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:23.169677 16050 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:23.170130 16050 webserver.cc:460] Webserver started at http://127.11.114.193:42399/ using document root <none> and password file <none>
I20260807 06:38:23.170397 16050 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:23.170473 16050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:23.170643 16050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:23.171769 16050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "e8d908e2525543a6a0cdd1bba930a35d"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "4352e8ad8efb1c94584b1499a80d4498"
server_key_iv: "3fc0b36303d1b4e83094d136ee1c4421"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.172174 16050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "e8d908e2525543a6a0cdd1bba930a35d"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "4352e8ad8efb1c94584b1499a80d4498"
server_key_iv: "3fc0b36303d1b4e83094d136ee1c4421"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.173681 16050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:23.174592 16068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.174945 16050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:23.175060 16050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "e8d908e2525543a6a0cdd1bba930a35d"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "4352e8ad8efb1c94584b1499a80d4498"
server_key_iv: "3fc0b36303d1b4e83094d136ee1c4421"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.175153 16050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:23.195726 16050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:23.198024 16050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:23.198196 16050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:23.198457 16050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:23.198803 16050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:23.198864 16050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.198917 16050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:23.198944 16050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.205431 16050 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:40369
I20260807 06:38:23.205489 16225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:40369 every 8 connection(s)
I20260807 06:38:23.205905 16050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:23.208285 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 16050
I20260807 06:38:23.208424 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:23.208627 11723 external_mini_cluster.cc:1428] Setting key 6978c287a4d136be72613eb382276eb2
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084703, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:23.217625 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:23.206696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:33995 (local address 127.11.114.254:36757)
0807 06:38:23.206745 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:23.206748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:23.206953 (+ 205us) server_negotiation.cc:407] Connection header received
0807 06:38:23.207487 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:23.207488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:23.207501 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:23.207565 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:23.208708 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.209510 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.210608 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.210991 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.211082 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:23.214322 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:23.214324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:23.214324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:23.214335 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:23.216062 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.216543 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.216545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.216546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.216617 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.216889 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.216890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.216890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.217018 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:23.217096 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:23.217398 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 06:38:23.217500 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:38:23.217796 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:23.206641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:36757 (local address 127.11.114.193:33995)
0807 06:38:23.206812 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:23.206856 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:23.207397 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:23.207640 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:23.207643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:23.208302 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:23.208592 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:23.208598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.209655 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.209656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:23.210501 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:23.210504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.211132 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.211134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:23.211349 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:23.211975 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:23.211987 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:23.214193 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:23.216194 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:23.216202 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:23.216204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:23.216458 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:23.216696 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:23.216696 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:23.216697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:23.216816 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:23.217088 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:23.217095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:23.217252 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 06:38:23.217362 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 06:38:23.217532 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260807 06:38:23.218348 16227 heartbeater.cc:347] Connected to a master server at 127.11.114.254:36757
I20260807 06:38:23.218478 16227 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:23.218724 16227 heartbeater.cc:511] Master 127.11.114.254:36757 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
I20260807 06:38:23.219357 15970 ts_manager.cc:194] Registered new tserver with Master: e8d908e2525543a6a0cdd1bba930a35d (127.11.114.193:40369)
I20260807 06:38:23.220064 15970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:33995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:23.238709 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36757
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:23.325767 16233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:23.325970 16233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:23.325994 16233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:23.327137 16233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:23.327180 16233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:23.327207 16233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:23.328685 16233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:23.328944 16233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:23.329221 16233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:23.334646 16239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:23.334656 16243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:23.334652 16240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:23.335028 16233 server_base.cc:1034] running on GCE node
I20260807 06:38:23.335320 16233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:23.335584 16233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:23.336771 16233 hybrid_clock.cc:648] HybridClock initialized: now 1786084703336750 us; error 38 us; skew 500 ppm
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084703, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:23.339624 16233 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:23.340035 16233 webserver.cc:460] Webserver started at http://127.11.114.194:41677/ using document root <none> and password file <none>
I20260807 06:38:23.340255 16233 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:23.340301 16233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:23.340399 16233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:23.341466 16233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "d0b9bf99a3c24ca68019f817458288d5"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "76ad6ffc7b5b45c0205f4f466aa48318"
server_key_iv: "5c360fcd2ea4ae8aea306872b6481fc0"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.341890 16233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "d0b9bf99a3c24ca68019f817458288d5"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "76ad6ffc7b5b45c0205f4f466aa48318"
server_key_iv: "5c360fcd2ea4ae8aea306872b6481fc0"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.343271 16233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:23.344118 16254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.344301 16233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:23.344417 16233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "d0b9bf99a3c24ca68019f817458288d5"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "76ad6ffc7b5b45c0205f4f466aa48318"
server_key_iv: "5c360fcd2ea4ae8aea306872b6481fc0"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.344496 16233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:23.367650 16233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:23.370055 16233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:23.370244 16233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:23.370508 16233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:23.370827 16233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:23.370889 16233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.370934 16233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:23.370963 16233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.377029 16233 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:46233
I20260807 06:38:23.377084 16410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:46233 every 8 connection(s)
I20260807 06:38:23.377409 16233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084703, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:23.384433 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 16233
I20260807 06:38:23.384526 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:23.384768 11723 external_mini_cluster.cc:1428] Setting key 5c8745d651716fea0a75656c408ea932
I20260807 06:38:23.386996 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:23.378163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:60425 (local address 127.11.114.254:36757)
0807 06:38:23.378214 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:23.378216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:23.378438 (+ 222us) server_negotiation.cc:407] Connection header received
0807 06:38:23.378894 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:23.378896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:23.378906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:23.378994 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:23.379960 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.380469 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.381466 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.381771 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.381837 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:23.383883 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:23.383884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:23.383885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:23.383891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:23.385418 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.385931 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.385933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.385933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.385994 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.386268 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.386269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.386269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.386424 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:23.386496 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:23.386816 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 06:38:23.386905 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:38:23.387157 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:23.378148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:36757 (local address 127.11.114.194:60425)
0807 06:38:23.378314 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:23.378349 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:23.378814 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:23.379079 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:23.379083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:23.379690 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:23.379880 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:23.379884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.380548 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.380549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:23.381387 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:23.381390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.381857 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.381858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:23.381970 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:23.382483 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:23.382491 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:23.383802 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:23.385513 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:23.385519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:23.385522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:23.385840 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:23.386067 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:23.386067 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:23.386068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:23.386192 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:23.386500 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:23.386506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:23.386650 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 06:38:23.386762 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:38:23.386918 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260807 06:38:23.387719 16411 heartbeater.cc:347] Connected to a master server at 127.11.114.254:36757
I20260807 06:38:23.387828 16411 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:23.388000 16411 heartbeater.cc:511] Master 127.11.114.254:36757 requested a full tablet report, sending...
I20260807 06:38:23.388424 15970 ts_manager.cc:194] Registered new tserver with Master: d0b9bf99a3c24ca68019f817458288d5 (127.11.114.194:46233)
I20260807 06:38:23.388803 15970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:60425
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:23.413288 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36757
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:23.496630 16417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:23.496891 16417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:23.496932 16417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:23.498107 16417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:23.498181 16417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:23.498234 16417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:23.499760 16417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:23.500033 16417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:23.500288 16417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:23.505635 16426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:23.505653 16425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:23.505659 16428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:23.506233 16417 server_base.cc:1034] running on GCE node
I20260807 06:38:23.506484 16417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:23.506734 16417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:23.507918 16417 hybrid_clock.cc:648] HybridClock initialized: now 1786084703507903 us; error 50 us; skew 500 ppm
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084703, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:23.510653 16417 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:23.511032 16417 webserver.cc:460] Webserver started at http://127.11.114.195:33673/ using document root <none> and password file <none>
I20260807 06:38:23.511298 16417 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:23.511346 16417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:23.511514 16417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:23.512598 16417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "3a7a94df226f40ab883fe8b7d38850f9"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "bab5c52109cedbdf023bd26f7d23d586"
server_key_iv: "c990bd6a6c9b06891b2fe55006aab6f2"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.512964 16417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "3a7a94df226f40ab883fe8b7d38850f9"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "bab5c52109cedbdf023bd26f7d23d586"
server_key_iv: "c990bd6a6c9b06891b2fe55006aab6f2"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.514506 16417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:23.515268 16437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.515451 16417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:23.515554 16417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "3a7a94df226f40ab883fe8b7d38850f9"
format_stamp: "Formatted at 2026-08-07 06:38:23 on dist-test-slave-gkw7"
server_key: "bab5c52109cedbdf023bd26f7d23d586"
server_key_iv: "c990bd6a6c9b06891b2fe55006aab6f2"
server_key_version: "encryptionkey@0"
I20260807 06:38:23.515632 16417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:23.552151 16417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:23.554414 16417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:23.554589 16417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:23.554864 16417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:23.555197 16417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:23.555234 16417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.555256 16417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:23.555269 16417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:23.561151 16417 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:46167
I20260807 06:38:23.561285 16600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:46167 every 8 connection(s)
I20260807 06:38:23.561676 16417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084703, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:23.569599 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 16417
I20260807 06:38:23.569703 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:23.569901 11723 external_mini_cluster.cc:1428] Setting key 909fef0b23e4f1f52811f8455709ffac
I20260807 06:38:23.571561 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:23.562476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:39201 (local address 127.11.114.254:36757)
0807 06:38:23.562576 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:23.562578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:23.562754 (+ 176us) server_negotiation.cc:407] Connection header received
0807 06:38:23.563320 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:23.563321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:23.563335 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:23.563387 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:23.564352 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.564864 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.565978 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.566192 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.566273 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:23.568565 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:23.568566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:23.568567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:23.568574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:23.570095 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.570603 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.570605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.570605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.570666 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.570946 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.570947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.570947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.571081 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:23.571135 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:23.571420 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 06:38:23.571496 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:38:23.571692 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:23.562433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:36757 (local address 127.11.114.195:39201)
0807 06:38:23.562623 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:23.562673 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:23.563199 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:23.563457 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:23.563461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:23.564053 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:23.564247 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:23.564251 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.564964 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.564966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:23.565886 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:23.565889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.566280 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.566282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:23.566409 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:23.566883 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:23.566891 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:23.568454 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:23.570170 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:23.570177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:23.570179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:23.570527 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:23.570725 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:23.570726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:23.570726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:23.570869 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:23.571144 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:23.571149 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:23.571267 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 06:38:23.571354 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:38:23.571476 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260807 06:38:23.572189 16601 heartbeater.cc:347] Connected to a master server at 127.11.114.254:36757
I20260807 06:38:23.572309 16601 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:23.572507 16601 heartbeater.cc:511] Master 127.11.114.254:36757 requested a full tablet report, sending...
I20260807 06:38:23.572942 15970 ts_manager.cc:194] Registered new tserver with Master: 3a7a94df226f40ab883fe8b7d38850f9 (127.11.114.195:46167)
I20260807 06:38:23.573323 15970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:39201
I20260807 06:38:23.582064 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084703, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[15901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084703, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:23.606832 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:23.599144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35244 (local address 127.11.114.254:36757)
0807 06:38:23.599239 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:23.599241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:23.599338 (+ 97us) server_negotiation.cc:407] Connection header received
0807 06:38:23.599391 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:23.599391 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:23.599401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:23.599465 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:23.600053 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.600560 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.601310 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:23.601501 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:23.601642 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:23.604177 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:23.604179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:23.604179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:23.604188 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:23.605711 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.606043 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.606044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.606044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.606104 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:23.606317 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:23.606318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:23.606319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:23.606492 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:23.606583 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:23.606693 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 06:38:23.606761 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 06:38:23.607134 15970 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35244
I20260807 06:38:23.607621 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 16050
I20260807 06:38:23.620738 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 16233
I20260807 06:38:23.647230 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 16417
I20260807 06:38:23.655932 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 15919
2026-08-07T06:38:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2165 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[16617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[16617](info): set up 2 sockets
Aug 07 06:38:23 dist-test-slave-gkw7 krb5kdc[16617](info): commencing operation
krb5kdc: starting...
W20260807 06:38:24.932174 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:25.060953 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:45405
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:45405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:25.145823 16632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:25.146050 16632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:25.146076 16632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:25.147200 16632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:25.147243 16632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:25.147259 16632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:25.147274 16632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:25.147287 16632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:25.148799 16632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:45405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:45405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:25.149013 16632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:25.149204 16632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:25.154250 16642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:25.154240 16638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:25.154379 16640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:25.154629 16632 server_base.cc:1034] running on GCE node
I20260807 06:38:25.154919 16632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:25.155236 16632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:25.156418 16632 hybrid_clock.cc:648] HybridClock initialized: now 1786084705156390 us; error 49 us; skew 500 ppm
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:25.159173 16632 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:25.159670 16632 webserver.cc:460] Webserver started at http://127.11.114.254:45063/ using document root <none> and password file <none>
I20260807 06:38:25.159880 16632 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:25.159926 16632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:25.160024 16632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:25.161089 16632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "8458175f464544149c38e0f2d77588a9"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "8b3121e7d1e4be6e5cced4f75983e117"
server_key_iv: "77d3c8bba8d48f27e5ea47043636518a"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.161438 16632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "8458175f464544149c38e0f2d77588a9"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "8b3121e7d1e4be6e5cced4f75983e117"
server_key_iv: "77d3c8bba8d48f27e5ea47043636518a"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.162962 16632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:25.163769 16651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.164028 16632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:25.164129 16632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "8458175f464544149c38e0f2d77588a9"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "8b3121e7d1e4be6e5cced4f75983e117"
server_key_iv: "77d3c8bba8d48f27e5ea47043636518a"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.164193 16632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:25.181375 16632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:25.183702 16632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:25.183833 16632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:25.188474 16632 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:45405
I20260807 06:38:25.188509 16717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:45405 every 8 connection(s)
I20260807 06:38:25.189059 16632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:25.189839 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:25.192103 16718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9: Bootstrap starting.
I20260807 06:38:25.192706 16718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:25.193094 16718 log.cc:824] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:25.193981 16718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9: No bootstrap required, opened a new log
I20260807 06:38:25.196624 16718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8458175f464544149c38e0f2d77588a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 45405 } }
I20260807 06:38:25.196749 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 16632
I20260807 06:38:25.196801 16718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:25.196870 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:25.196856 16718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8458175f464544149c38e0f2d77588a9, State: Initialized, Role: FOLLOWER
I20260807 06:38:25.197057 11723 external_mini_cluster.cc:1428] Setting key a11b0bcdfbce944476e4fedd73a9cb3d
I20260807 06:38:25.197037 16718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8458175f464544149c38e0f2d77588a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 45405 } }
I20260807 06:38:25.197131 16718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:25.197173 16718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:25.197227 16718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:25.198076 16718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8458175f464544149c38e0f2d77588a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 45405 } }
I20260807 06:38:25.198203 16718 leader_election.cc:302] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8458175f464544149c38e0f2d77588a9; no voters:
I20260807 06:38:25.198443 16718 leader_election.cc:288] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:25.198473 16724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:25.198601 16724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [term 1 LEADER]: Becoming Leader. State: Replica: 8458175f464544149c38e0f2d77588a9, State: Running, Role: LEADER
I20260807 06:38:25.198724 16718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:25.198756 16724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8458175f464544149c38e0f2d77588a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 45405 } }
I20260807 06:38:25.199132 16724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8458175f464544149c38e0f2d77588a9. Latest consensus state: current_term: 1 leader_uuid: "8458175f464544149c38e0f2d77588a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8458175f464544149c38e0f2d77588a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 45405 } } }
I20260807 06:38:25.199147 16727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8458175f464544149c38e0f2d77588a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8458175f464544149c38e0f2d77588a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 45405 } } }
I20260807 06:38:25.199344 16727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:25.199576 16724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:25.199898 16735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:25.200505 16735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:25.202167 16735 catalog_manager.cc:1347] Generated new cluster ID: 65909bddfe344878aaea69ab7567bd4d
I20260807 06:38:25.202222 16735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:25.209946 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.198125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44088 (local address 127.11.114.254:45405)
0807 06:38:25.198309 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:25.198318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:25.198354 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:38:25.198459 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:25.198464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:25.198534 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:25.198701 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:25.199633 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.201138 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.202213 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.202838 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.202956 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.205990 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:25.205994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:25.205995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:25.206007 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:25.208144 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.208567 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.208571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.208571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.208689 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.208913 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.208914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.208914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.209244 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:25.209410 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:25.209536 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 06:38:25.209723 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260807 06:38:25.211158 16735 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:25.211838 16735 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:25.218717 16735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8458175f464544149c38e0f2d77588a9: Generated new TSK 0
I20260807 06:38:25.218886 16735 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:25.248010 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:45405
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:25.337162 16752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:25.337419 16752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:25.337464 16752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:25.338683 16752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:25.338763 16752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:25.338812 16752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:25.340361 16752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:25.340648 16752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:25.340893 16752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:25.346261 16760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:25.346354 16759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:25.346269 16752 server_base.cc:1034] running on GCE node
W20260807 06:38:25.346271 16764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:25.346691 16752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:25.346939 16752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:25.348105 16752 hybrid_clock.cc:648] HybridClock initialized: now 1786084705348099 us; error 41 us; skew 500 ppm
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:25.350742 16752 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:25.351194 16752 webserver.cc:460] Webserver started at http://127.11.114.193:46007/ using document root <none> and password file <none>
I20260807 06:38:25.351423 16752 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:25.351495 16752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:25.351648 16752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:25.352684 16752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "932329bcd7b34ea0a0a7e13d4d599e42"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "8f21d39aff51254c6de5796c4b5b41f9"
server_key_iv: "8437a5f3103331ebd15bcc134e7fc50a"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.353050 16752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "932329bcd7b34ea0a0a7e13d4d599e42"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "8f21d39aff51254c6de5796c4b5b41f9"
server_key_iv: "8437a5f3103331ebd15bcc134e7fc50a"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.354552 16752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:25.355386 16774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.355602 16752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:25.355711 16752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "932329bcd7b34ea0a0a7e13d4d599e42"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "8f21d39aff51254c6de5796c4b5b41f9"
server_key_iv: "8437a5f3103331ebd15bcc134e7fc50a"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.355829 16752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:25.378898 16752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:25.381091 16752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:25.381276 16752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:25.381580 16752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:25.381946 16752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:25.382002 16752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.382045 16752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:25.382074 16752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.387900 16752 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:35225
I20260807 06:38:25.387977 16923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:35225 every 8 connection(s)
I20260807 06:38:25.388321 16752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:25.394140 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 16752
I20260807 06:38:25.394263 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:25.394464 11723 external_mini_cluster.cc:1428] Setting key a50bf9b0d57b0f6647cf534661716bd3
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:25.398293 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.389028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:45405 (local address 127.11.114.193:48609)
0807 06:38:25.389207 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:25.389249 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:25.390261 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:25.390499 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:25.390502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:25.390757 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:25.390950 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:25.390954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.391602 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.391603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:25.392401 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:25.392403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.392754 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.392755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:25.392853 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.393393 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:25.393402 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:25.394994 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:25.396995 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:25.397001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:25.397003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:25.397223 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:25.397430 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:25.397431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:25.397431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:25.397515 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:25.397823 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:25.397827 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:25.397939 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 06:38:25.398014 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 06:38:25.398134 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260807 06:38:25.398331 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.389086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:48609 (local address 127.11.114.254:45405)
0807 06:38:25.389160 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:25.389163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:25.389340 (+ 177us) server_negotiation.cc:407] Connection header received
0807 06:38:25.390368 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:25.390369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:25.390384 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:25.390445 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:25.391026 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.391522 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.392482 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.392682 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.392736 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.395193 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:25.395194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:25.395195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:25.395203 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:25.396907 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.397294 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.397296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.397296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.397365 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.397610 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.397611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.397612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.397759 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:25.397832 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:25.398095 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 06:38:25.398197 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:38:25.398862 16925 heartbeater.cc:347] Connected to a master server at 127.11.114.254:45405
I20260807 06:38:25.398964 16925 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:25.399176 16925 heartbeater.cc:511] Master 127.11.114.254:45405 requested a full tablet report, sending...
I20260807 06:38:25.399739 16677 ts_manager.cc:194] Registered new tserver with Master: 932329bcd7b34ea0a0a7e13d4d599e42 (127.11.114.193:35225)
I20260807 06:38:25.400409 16677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:48609
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:25.422231 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:45405
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:25.504882 16933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:25.505103 16933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:25.505128 16933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:25.506294 16933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:25.506336 16933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:25.506362 16933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:25.507925 16933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:25.508186 16933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:25.508435 16933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:25.514101 16943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:25.514104 16942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:25.514133 16933 server_base.cc:1034] running on GCE node
W20260807 06:38:25.514160 16946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:25.514598 16933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:25.514909 16933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:25.516114 16933 hybrid_clock.cc:648] HybridClock initialized: now 1786084705516098 us; error 27 us; skew 500 ppm
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:25.518962 16933 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:25.519464 16933 webserver.cc:460] Webserver started at http://127.11.114.194:39817/ using document root <none> and password file <none>
I20260807 06:38:25.519743 16933 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:25.519821 16933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:25.519984 16933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:25.521293 16933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "ff48086318414caabf7579f631ec874b"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "cfe5e13a4509bc58421c320892c49acf"
server_key_iv: "b713782f62a9e83bf8c7a117b7aa69d1"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.521700 16933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "ff48086318414caabf7579f631ec874b"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "cfe5e13a4509bc58421c320892c49acf"
server_key_iv: "b713782f62a9e83bf8c7a117b7aa69d1"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.523087 16933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:25.523835 16954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.524003 16933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:25.524112 16933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "ff48086318414caabf7579f631ec874b"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "cfe5e13a4509bc58421c320892c49acf"
server_key_iv: "b713782f62a9e83bf8c7a117b7aa69d1"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.524186 16933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:25.532553 16933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:25.534616 16933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:25.534780 16933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:25.535010 16933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:25.535327 16933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:25.535383 16933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.535421 16933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:25.535449 16933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.541219 16933 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:46709
I20260807 06:38:25.541261 17108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:46709 every 8 connection(s)
I20260807 06:38:25.541663 16933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:25.547744 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 16933
I20260807 06:38:25.547858 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:25.548040 11723 external_mini_cluster.cc:1428] Setting key e5cfcb106f23967268361822b8eeb0e5
I20260807 06:38:25.551075 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.542446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:60135 (local address 127.11.114.254:45405)
0807 06:38:25.542496 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:25.542499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:25.542707 (+ 208us) server_negotiation.cc:407] Connection header received
0807 06:38:25.543613 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:25.543615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:25.543627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:25.543694 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:25.544258 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.544794 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.545870 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.546060 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.546143 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.548144 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:25.548146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:25.548146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:25.548159 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:25.549620 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.550019 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.550020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.550020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.550082 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.550346 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.550347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.550347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.550462 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:25.550533 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:25.550848 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 06:38:25.550950 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:38:25.551162 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.542426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:45405 (local address 127.11.114.194:60135)
0807 06:38:25.542579 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:25.542619 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:25.543516 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:25.543750 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:25.543753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:25.544000 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:25.544186 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:25.544191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.544875 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.544875 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:25.545787 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:25.545789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.546150 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.546151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:25.546264 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.546700 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:25.546707 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:25.548021 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:25.549698 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:25.549705 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:25.549707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:25.549938 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:25.550149 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:25.550150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:25.550151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:25.550274 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:25.550535 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:25.550542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:25.550680 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 06:38:25.550798 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 06:38:25.550965 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260807 06:38:25.551628 17110 heartbeater.cc:347] Connected to a master server at 127.11.114.254:45405
I20260807 06:38:25.551757 17110 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:25.551939 17110 heartbeater.cc:511] Master 127.11.114.254:45405 requested a full tablet report, sending...
I20260807 06:38:25.552378 16677 ts_manager.cc:194] Registered new tserver with Master: ff48086318414caabf7579f631ec874b (127.11.114.194:46709)
I20260807 06:38:25.552760 16677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:60135
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:25.575762 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:45405
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:25.659611 17121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:25.659818 17121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:25.659842 17121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:25.661072 17121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:25.661118 17121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:25.661146 17121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:25.662672 17121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:25.662919 17121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:25.663161 17121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:25.668251 17130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:25.668327 17127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:25.668247 17128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:25.668963 17121 server_base.cc:1034] running on GCE node
I20260807 06:38:25.669168 17121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:25.669401 17121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:25.670608 17121 hybrid_clock.cc:648] HybridClock initialized: now 1786084705670580 us; error 51 us; skew 500 ppm
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:25.673436 17121 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:25.673965 17121 webserver.cc:460] Webserver started at http://127.11.114.195:45367/ using document root <none> and password file <none>
I20260807 06:38:25.674206 17121 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:25.674278 17121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:25.674433 17121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:25.675525 17121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "d66673f3614346c09d8fea9c25deb5a1"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "9b698ce700434e25efab1d8e159c7ce3"
server_key_iv: "3a2f8bb5bdff116b37d40003adfe1b4d"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.675910 17121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "d66673f3614346c09d8fea9c25deb5a1"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "9b698ce700434e25efab1d8e159c7ce3"
server_key_iv: "3a2f8bb5bdff116b37d40003adfe1b4d"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.677381 17121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:25.678253 17143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.678489 17121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:25.678628 17121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "d66673f3614346c09d8fea9c25deb5a1"
format_stamp: "Formatted at 2026-08-07 06:38:25 on dist-test-slave-gkw7"
server_key: "9b698ce700434e25efab1d8e159c7ce3"
server_key_iv: "3a2f8bb5bdff116b37d40003adfe1b4d"
server_key_version: "encryptionkey@0"
I20260807 06:38:25.678709 17121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:25.689009 17121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:25.691399 17121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:25.691572 17121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:25.691823 17121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:25.692147 17121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:25.692200 17121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.692242 17121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:25.692271 17121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:25.698508 17121 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:35677
I20260807 06:38:25.698607 17302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:35677 every 8 connection(s)
I20260807 06:38:25.698926 17121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:38:25.701191 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 17121
I20260807 06:38:25.701313 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:25.701512 11723 external_mini_cluster.cc:1428] Setting key b143a6cd2a69640fc58137a43fb656c9
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:25.708918 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.699756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:55205 (local address 127.11.114.254:45405)
0807 06:38:25.699826 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:25.699828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:25.700036 (+ 208us) server_negotiation.cc:407] Connection header received
0807 06:38:25.701169 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:25.701171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:25.701186 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:25.701292 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:25.702197 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.702828 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.703847 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.704037 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.704103 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.706339 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:25.706340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:25.706340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:25.706347 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:25.707550 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.707913 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.707914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.707915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.707964 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.708233 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.708235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.708235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.708358 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:25.708426 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:25.708770 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 06:38:25.708855 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:38:25.709185 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.699757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:45405 (local address 127.11.114.195:55205)
0807 06:38:25.699920 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:25.699957 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:25.701014 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:25.701366 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:25.701371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:25.701822 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:25.702110 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:25.702116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.702912 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.702913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:25.703747 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:25.703749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.704129 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.704130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:25.704232 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.704788 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:25.704797 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:25.706247 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:25.707623 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:25.707629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:25.707630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:25.707848 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:25.708021 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:25.708022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:25.708022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:25.708150 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:25.708453 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:25.708460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:25.708616 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 06:38:25.708735 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 06:38:25.708874 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260807 06:38:25.709831 17303 heartbeater.cc:347] Connected to a master server at 127.11.114.254:45405
I20260807 06:38:25.709956 17303 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:25.710151 17303 heartbeater.cc:511] Master 127.11.114.254:45405 requested a full tablet report, sending...
I20260807 06:38:25.710594 16677 ts_manager.cc:194] Registered new tserver with Master: d66673f3614346c09d8fea9c25deb5a1 (127.11.114.195:35677)
I20260807 06:38:25.710999 16677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:55205
I20260807 06:38:25.713430 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:25.737790 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.729720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.11.114.193:35225)
0807 06:38:25.729874 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:25.729879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:25.729898 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:38:25.729926 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:25.729928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:25.729942 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:25.730017 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:25.730659 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.731375 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.732183 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.732518 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.732600 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.734995 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:25.735003 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:25.735004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:25.735010 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:25.736572 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.736944 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.736947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.736948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.737011 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.737212 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.737212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.737213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.737348 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:25.737454 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:25.737605 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 06:38:25.737673 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[16617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:25.745923 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:25.739278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44090 (local address 127.11.114.254:45405)
0807 06:38:25.739354 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:25.739356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:25.739366 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:25.739399 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:25.739400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:25.739410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:25.739465 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:25.739891 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.740420 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.741082 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:25.741276 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:25.741335 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:25.743582 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:25.743583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:25.743583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:25.743590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:25.744836 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.745145 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.745146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.745146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.745200 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:25.745368 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:25.745368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:25.745369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:25.745510 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:25.745674 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:25.745782 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 06:38:25.745854 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 06:38:25.746191 16677 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44090
I20260807 06:38:25.746665 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 16752
I20260807 06:38:25.760735 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 16933
I20260807 06:38:25.784373 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 17121
I20260807 06:38:25.790745 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 16632
2026-08-07T06:38:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2132 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[17318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[17318](info): set up 2 sockets
Aug 07 06:38:25 dist-test-slave-gkw7 krb5kdc[17318](info): commencing operation
krb5kdc: starting...
W20260807 06:38:27.008891 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:27.136143 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33505
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:33505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:27.220605 17336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:27.220845 17336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:27.220885 17336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:27.222119 17336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:27.222183 17336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:27.222208 17336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:27.222232 17336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:27.222256 17336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:27.223775 17336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:33505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:27.224042 17336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:27.224268 17336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:27.229223 17348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:27.229295 17336 server_base.cc:1034] running on GCE node
W20260807 06:38:27.229197 17350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:27.229197 17344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:27.229696 17336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:27.229992 17336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:27.231194 17336 hybrid_clock.cc:648] HybridClock initialized: now 1786084707231170 us; error 38 us; skew 500 ppm
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:27.245028 17336 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:27.245388 17336 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:27.245431 17336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:27.245597 17336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:27.246680 17336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "a5882f7dafdc4a0187a6aa6b3314ab84"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "d54af5cb6ab3caad252987676f1f2934"
server_key_iv: "31fcdbad54fcd011135b273385c65bfb"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.247021 17336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "a5882f7dafdc4a0187a6aa6b3314ab84"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "d54af5cb6ab3caad252987676f1f2934"
server_key_iv: "31fcdbad54fcd011135b273385c65bfb"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.248355 17336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:27.249090 17357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.249276 17336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:27.249369 17336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "a5882f7dafdc4a0187a6aa6b3314ab84"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "d54af5cb6ab3caad252987676f1f2934"
server_key_iv: "31fcdbad54fcd011135b273385c65bfb"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.249423 17336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:27.293344 17336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:27.295635 17336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:27.295766 17336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:27.299857 17336 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:33505
I20260807 06:38:27.299955 17431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:33505 every 8 connection(s)
I20260807 06:38:27.300253 17336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:27.301422 17433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:27.302099 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 17336
I20260807 06:38:27.302227 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:27.302424 11723 external_mini_cluster.cc:1428] Setting key ff60dfe14099e0870f03ad4d4535031e
I20260807 06:38:27.304354 17433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84: Bootstrap starting.
I20260807 06:38:27.304986 17433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:27.305285 17433 log.cc:824] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:27.306016 17433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84: No bootstrap required, opened a new log
I20260807 06:38:27.307775 17433 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5882f7dafdc4a0187a6aa6b3314ab84" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33505 } }
I20260807 06:38:27.307937 17433 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:27.307989 17433 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5882f7dafdc4a0187a6aa6b3314ab84, State: Initialized, Role: FOLLOWER
I20260807 06:38:27.308156 17433 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5882f7dafdc4a0187a6aa6b3314ab84" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33505 } }
I20260807 06:38:27.308256 17433 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:27.308301 17433 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:27.308350 17433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:27.308887 17433 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5882f7dafdc4a0187a6aa6b3314ab84" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33505 } }
I20260807 06:38:27.309010 17433 leader_election.cc:302] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5882f7dafdc4a0187a6aa6b3314ab84; no voters:
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:27.309206 17433 leader_election.cc:288] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:27.309273 17442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:27.309424 17442 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [term 1 LEADER]: Becoming Leader. State: Replica: a5882f7dafdc4a0187a6aa6b3314ab84, State: Running, Role: LEADER
I20260807 06:38:27.309514 17433 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:27.309556 17442 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5882f7dafdc4a0187a6aa6b3314ab84" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33505 } }
I20260807 06:38:27.309890 17443 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5882f7dafdc4a0187a6aa6b3314ab84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5882f7dafdc4a0187a6aa6b3314ab84" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33505 } } }
I20260807 06:38:27.309989 17443 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:27.310192 17444 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5882f7dafdc4a0187a6aa6b3314ab84. Latest consensus state: current_term: 1 leader_uuid: "a5882f7dafdc4a0187a6aa6b3314ab84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5882f7dafdc4a0187a6aa6b3314ab84" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33505 } } }
I20260807 06:38:27.310258 17444 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:27.310286 17447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:27.312196 17447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:27.313702 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.302991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.11.114.254:33505)
0807 06:38:27.303217 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.303225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.303252 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:38:27.303648 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.303654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.303699 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.303854 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:27.304554 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.305632 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.306481 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.306910 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.307032 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.309767 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:27.309772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:27.309773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:27.309789 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:27.311692 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.312171 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.312175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.312176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.312315 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.312574 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.312575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.312576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.312871 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:27.313010 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.313313 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.313458 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":115,"threads_started":1}
I20260807 06:38:27.314150 17447 catalog_manager.cc:1347] Generated new cluster ID: 9a6388cf6df04fa1bf6ad0f0b01825e2
I20260807 06:38:27.314199 17447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:27.322523 17447 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:27.323088 17447 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:27.328557 17447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84: Generated new TSK 0
I20260807 06:38:27.328753 17447 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:27.357000 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33505
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:27.439919 17466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:27.440135 17466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:27.440160 17466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:27.441241 17466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:27.441275 17466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:27.441301 17466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:27.442843 17466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:27.443027 17466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:27.443204 17466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:27.448174 17476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:27.448150 17475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:27.448165 17478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:27.448383 17466 server_base.cc:1034] running on GCE node
I20260807 06:38:27.448575 17466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:27.448772 17466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:27.449908 17466 hybrid_clock.cc:648] HybridClock initialized: now 1786084707449897 us; error 28 us; skew 500 ppm
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:27.452523 17466 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:27.452903 17466 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:27.452952 17466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:27.453042 17466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:27.454213 17466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "e6ec0d21b84244b78b8912cb7d43f7cb"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "aa36a7f1d725f10f023cfd8b46a7d517"
server_key_iv: "dbe8d05f35885bf618ff3bc2cc4bd74d"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.454520 17466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "e6ec0d21b84244b78b8912cb7d43f7cb"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "aa36a7f1d725f10f023cfd8b46a7d517"
server_key_iv: "dbe8d05f35885bf618ff3bc2cc4bd74d"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.455803 17466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:27.456545 17486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.456768 17466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:27.456840 17466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "e6ec0d21b84244b78b8912cb7d43f7cb"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "aa36a7f1d725f10f023cfd8b46a7d517"
server_key_iv: "dbe8d05f35885bf618ff3bc2cc4bd74d"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.456892 17466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:27.467038 17466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:27.469288 17466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:27.469420 17466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:27.469676 17466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:27.470012 17466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:27.470047 17466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.470069 17466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:27.470118 17466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.476317 17466 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:42565
I20260807 06:38:27.476359 17638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:42565 every 8 connection(s)
I20260807 06:38:27.476619 17466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:27.482280 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 17466
I20260807 06:38:27.482402 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:27.482632 11723 external_mini_cluster.cc:1428] Setting key 801c8ddbfd0fdb252816d7a16c8dff3d
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:27.487246 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.477373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:47927 (local address 127.11.114.254:33505)
0807 06:38:27.477480 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.477482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.477691 (+ 209us) server_negotiation.cc:407] Connection header received
0807 06:38:27.478219 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.478220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.478234 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.478283 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:27.479303 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.479814 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.480832 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.481049 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.481129 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.483956 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:27.483957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:27.483957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:27.483964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:27.485744 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.486191 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.486192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.486193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.486271 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.486522 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.486523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.486523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.486679 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:27.486761 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.487009 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.487108 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:38:27.487267 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.477315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33505 (local address 127.11.114.193:47927)
0807 06:38:27.477513 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:27.477592 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:27.478106 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:27.478365 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:27.478369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:27.478627 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:27.479187 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.479193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.479913 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.479914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.480735 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.480737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.481136 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.481137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.481273 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.481886 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:27.481895 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:27.483865 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:27.485854 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:27.485861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:27.485863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:27.486100 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:27.486345 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:27.486346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:27.486346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:27.486454 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:27.486748 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:27.486753 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:27.486876 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 06:38:27.486962 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 06:38:27.487095 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260807 06:38:27.487775 17639 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33505
I20260807 06:38:27.487882 17639 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:27.488082 17639 heartbeater.cc:511] Master 127.11.114.254:33505 requested a full tablet report, sending...
I20260807 06:38:27.488651 17381 ts_manager.cc:194] Registered new tserver with Master: e6ec0d21b84244b78b8912cb7d43f7cb (127.11.114.193:42565)
I20260807 06:38:27.489285 17381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:47927
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:27.512501 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33505
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:27.598625 17647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:27.598889 17647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:27.598928 17647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:27.600088 17647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:27.600167 17647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:27.600212 17647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:27.601835 17647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:27.602119 17647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:27.602353 17647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:27.607420 17660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:27.607488 17652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:27.607560 17647 server_base.cc:1034] running on GCE node
W20260807 06:38:27.607496 17655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:27.607897 17647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:27.608103 17647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:27.609241 17647 hybrid_clock.cc:648] HybridClock initialized: now 1786084707609229 us; error 33 us; skew 500 ppm
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:27.612118 17647 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:27.612506 17647 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:27.612556 17647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:27.612739 17647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:27.613936 17647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "76504f4822594df6a7d955e4fa06706d"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "97261d9f40ef816afce5d225bd5bd829"
server_key_iv: "f84da31d5a15dc844a5f0a9f6d7e6bea"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.614302 17647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "76504f4822594df6a7d955e4fa06706d"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "97261d9f40ef816afce5d225bd5bd829"
server_key_iv: "f84da31d5a15dc844a5f0a9f6d7e6bea"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.615623 17647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 06:38:27.616528 17668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.616770 17647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:27.616868 17647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "76504f4822594df6a7d955e4fa06706d"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "97261d9f40ef816afce5d225bd5bd829"
server_key_iv: "f84da31d5a15dc844a5f0a9f6d7e6bea"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.616923 17647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:27.637072 17647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:27.639333 17647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:27.639453 17647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:27.639662 17647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:27.640014 17647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:27.640050 17647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.640074 17647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:27.640087 17647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.646621 17647 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:36991
I20260807 06:38:27.646723 17823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:36991 every 8 connection(s)
I20260807 06:38:27.646984 17647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:27.648584 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 17647
I20260807 06:38:27.648701 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:27.648924 11723 external_mini_cluster.cc:1428] Setting key bd0c37b56ac5ab40d6cff80f9771f203
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
I20260807 06:38:27.658195 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.647790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:46427 (local address 127.11.114.254:33505)
0807 06:38:27.647855 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.647857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.648100 (+ 243us) server_negotiation.cc:407] Connection header received
0807 06:38:27.648607 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.648608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.648622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.648697 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:27.650181 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.650665 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.652122 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.652329 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.652421 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.654957 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:27.654959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:27.654960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:27.654968 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:27.656657 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.657119 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.657121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.657122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.657189 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.657451 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.657452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.657452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.657641 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:27.657720 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.658000 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.658098 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:38:27.658391 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.647773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33505 (local address 127.11.114.194:46427)
0807 06:38:27.647959 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:27.648003 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:27.648492 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:27.648819 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:27.648825 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:27.649251 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:27.650040 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.650048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.650759 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.650760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.652014 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.652017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.652430 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.652432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.652565 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.653163 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:27.653171 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:27.654853 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:27.656777 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:27.656785 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:27.656787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:27.657025 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:27.657260 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:27.657261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:27.657262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:27.657373 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:27.657711 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:27.657718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:27.657863 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 06:38:27.657978 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 06:38:27.658142 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260807 06:38:27.658967 17824 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33505
I20260807 06:38:27.659089 17824 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:27.659257 17824 heartbeater.cc:511] Master 127.11.114.254:33505 requested a full tablet report, sending...
I20260807 06:38:27.659685 17381 ts_manager.cc:194] Registered new tserver with Master: 76504f4822594df6a7d955e4fa06706d (127.11.114.194:36991)
I20260807 06:38:27.660036 17381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:46427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:27.676303 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33505
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:27.759331 17832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:27.759559 17832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:27.759600 17832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:27.760809 17832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:27.760875 17832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:27.760921 17832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:27.762507 17832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:27.762770 17832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:27.762995 17832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:27.768079 17840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:27.768085 17838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:27.768100 17845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:27.768339 17832 server_base.cc:1034] running on GCE node
I20260807 06:38:27.768554 17832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:27.768829 17832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:27.769980 17832 hybrid_clock.cc:648] HybridClock initialized: now 1786084707769965 us; error 45 us; skew 500 ppm
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:27.772948 17832 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:27.773312 17832 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:27.773389 17832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:27.773597 17832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:27.774695 17832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "3c2771a22d5049ca8792b780d78b8043"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "43d220eb6ed09d3d6ac7bb6c9ec56fd2"
server_key_iv: "5ea906d753bafbaaaa43ad03813b9903"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.775084 17832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "3c2771a22d5049ca8792b780d78b8043"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "43d220eb6ed09d3d6ac7bb6c9ec56fd2"
server_key_iv: "5ea906d753bafbaaaa43ad03813b9903"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.776393 17832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:27.777096 17853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.777258 17832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:27.777366 17832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "3c2771a22d5049ca8792b780d78b8043"
format_stamp: "Formatted at 2026-08-07 06:38:27 on dist-test-slave-gkw7"
server_key: "43d220eb6ed09d3d6ac7bb6c9ec56fd2"
server_key_iv: "5ea906d753bafbaaaa43ad03813b9903"
server_key_version: "encryptionkey@0"
I20260807 06:38:27.777439 17832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:27.793129 17832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:27.795074 17832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:27.795243 17832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:27.795492 17832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:27.795828 17832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:27.795884 17832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.795929 17832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:27.795953 17832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:27.802110 17832 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:44663
I20260807 06:38:27.802161 18007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:44663 every 8 connection(s)
I20260807 06:38:27.802454 17832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:38:27 dist-test-slave-gkw7 krb5kdc[17318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084707, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:27.811857 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.803276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:49941 (local address 127.11.114.254:33505)
0807 06:38:27.803324 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.803326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.803508 (+ 182us) server_negotiation.cc:407] Connection header received
0807 06:38:27.804132 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.804133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.804142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.804202 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:27.805120 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.805696 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.806693 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.806887 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.806944 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.809037 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:27.809038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:27.809039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:27.809045 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:27.810559 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.810942 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.810943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.810944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.810995 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.811296 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.811296 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.811297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.811403 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:27.811452 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.811711 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.811791 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:27.811965 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 17832
I20260807 06:38:27.812076 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:27.812124 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.803239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33505 (local address 127.11.114.195:49941)
0807 06:38:27.803392 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:27.803435 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:27.804038 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:27.804256 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:27.804259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:27.804519 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:27.805037 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.805042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.805767 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.805768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.806608 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.806611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.806981 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.806983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.807138 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.807620 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:27.807629 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:27.808951 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:27.810644 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:27.810650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:27.810652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:27.810873 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:27.811071 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:27.811072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:27.811072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:27.811235 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:27.811457 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:27.811462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:27.811595 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 06:38:27.811694 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 06:38:27.811862 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260807 06:38:27.812320 11723 external_mini_cluster.cc:1428] Setting key 69f80ac144fab71740ed9146b4ef45f8
I20260807 06:38:27.812772 18008 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33505
I20260807 06:38:27.812899 18008 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:27.813066 18008 heartbeater.cc:511] Master 127.11.114.254:33505 requested a full tablet report, sending...
I20260807 06:38:27.813501 17381 ts_manager.cc:194] Registered new tserver with Master: 3c2771a22d5049ca8792b780d78b8043 (127.11.114.195:44663)
I20260807 06:38:27.813946 17381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:49941
I20260807 06:38:27.824211 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:38:27.831641 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.825630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.11.114.254:33505)
0807 06:38:27.825684 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.825686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.825823 (+ 137us) server_negotiation.cc:407] Connection header received
0807 06:38:27.825864 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.825865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.825874 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.825957 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:27.826508 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.826981 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.827675 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.827856 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.827909 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.829269 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:27.829271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:27.829271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:27.829277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:27.830586 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.830898 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.830898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.830899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.830950 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:27.831118 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:27.831118 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:27.831119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:27.831256 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:27.831327 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.831522 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.831578 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 06:38:27.833516 17381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:27.835088 17381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:27.841621 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.837434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:44663 (local address 127.0.0.1:49872)
0807 06:38:27.837601 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:27.837646 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:27.837718 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:27.838023 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:27.838026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:27.838028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:27.838101 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.838103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.839253 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.839254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.840502 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.840505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.841275 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.841276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.841356 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.841369 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 06:38:27.841469 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:38:27.841547 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":62,"threads_started":1}
I20260807 06:38:27.841631 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.837526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49872 (local address 127.11.114.195:44663)
0807 06:38:27.837860 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.837871 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.837879 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:27.837911 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.837913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.837926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.838017 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:27.838196 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.839151 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.840585 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.841183 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.841279 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.841338 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.841488 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.841548 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":86,"threads_started":1}
I20260807 06:38:27.841980 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.837665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:36991 (local address 127.0.0.1:33368)
0807 06:38:27.838328 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:27.838338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:27.838388 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:27.838667 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:27.838667 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:27.838670 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:27.838719 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.838720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.839840 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.839841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.840777 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.840779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.841723 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.841724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.841811 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.841816 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:27.841875 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:38:27.841913 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":73,"threads_started":1}
I20260807 06:38:27.842200 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.837437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42565 (local address 127.0.0.1:35480)
0807 06:38:27.837785 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:27.837795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:27.837846 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:27.838213 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:27.838214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:27.838216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:27.838291 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.838293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.839576 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.839577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.840653 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.840657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.841633 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.841634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.842056 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.842061 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:27.842101 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 06:38:27.842143 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":106,"threads_started":1}
I20260807 06:38:27.842402 17918 tablet_service.cc:1467] Processing CreateTablet for tablet 23bccad48782400ab853c3f3ea09b47d (DEFAULT_TABLE table=test-table [id=8b63734f4cd74aa5a179e9bc3eeaa28a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:27.842590 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.837642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35480 (local address 127.11.114.193:42565)
0807 06:38:27.838054 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.838057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.838075 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:38:27.838108 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.838109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.838122 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.838216 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:27.838381 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.839474 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.840749 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.841488 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.842402 (+ 914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.842447 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.842469 (+ 22us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.842502 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":87,"threads_started":1}
I20260807 06:38:27.842850 17918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23bccad48782400ab853c3f3ea09b47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:27.843199 17552 tablet_service.cc:1467] Processing CreateTablet for tablet 23bccad48782400ab853c3f3ea09b47d (DEFAULT_TABLE table=test-table [id=8b63734f4cd74aa5a179e9bc3eeaa28a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:27.843523 17552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23bccad48782400ab853c3f3ea09b47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:27.845773 18034 tablet_bootstrap.cc:492] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: Bootstrap starting.
I20260807 06:38:27.845868 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.837849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.11.114.194:36991)
0807 06:38:27.837992 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.837997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.838520 (+ 523us) server_negotiation.cc:407] Connection header received
0807 06:38:27.838571 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.838573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.838589 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.838668 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:27.838778 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.839741 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.840860 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.841569 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.841635 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.841688 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.841908 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.845780 (+ 3872us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260807 06:38:27.846264 18036 tablet_bootstrap.cc:492] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb: Bootstrap starting.
I20260807 06:38:27.846457 18034 tablet_bootstrap.cc:654] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:27.846547 17733 tablet_service.cc:1467] Processing CreateTablet for tablet 23bccad48782400ab853c3f3ea09b47d (DEFAULT_TABLE table=test-table [id=8b63734f4cd74aa5a179e9bc3eeaa28a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:27.846881 17733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23bccad48782400ab853c3f3ea09b47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:27.846951 18036 tablet_bootstrap.cc:654] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:27.847198 18034 log.cc:824] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:27.847580 18036 log.cc:824] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:27.848065 18034 tablet_bootstrap.cc:492] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: No bootstrap required, opened a new log
I20260807 06:38:27.848143 18034 ts_tablet_manager.cc:1397] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:38:27.848335 18036 tablet_bootstrap.cc:492] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb: No bootstrap required, opened a new log
I20260807 06:38:27.848415 18036 ts_tablet_manager.cc:1397] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:38:27.850049 18036 raft_consensus.cc:348] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.850317 18039 tablet_bootstrap.cc:492] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Bootstrap starting.
I20260807 06:38:27.850353 18036 raft_consensus.cc:374] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:27.850404 18036 raft_consensus.cc:729] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ec0d21b84244b78b8912cb7d43f7cb, State: Initialized, Role: FOLLOWER
I20260807 06:38:27.850283 18034 raft_consensus.cc:348] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.850530 18036 consensus_queue.cc:260] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.850602 18034 raft_consensus.cc:374] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:27.850647 18034 raft_consensus.cc:729] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2771a22d5049ca8792b780d78b8043, State: Initialized, Role: FOLLOWER
I20260807 06:38:27.850781 18034 consensus_queue.cc:260] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.850842 18036 ts_tablet_manager.cc:1428] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:38:27.850971 18039 tablet_bootstrap.cc:654] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:27.850975 17639 heartbeater.cc:503] Master 127.11.114.254:33505 was elected leader, sending a full tablet report...
I20260807 06:38:27.851398 18034 ts_tablet_manager.cc:1428] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:38:27.851431 18008 heartbeater.cc:503] Master 127.11.114.254:33505 was elected leader, sending a full tablet report...
I20260807 06:38:27.851584 18039 log.cc:824] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:27.852579 18039 tablet_bootstrap.cc:492] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: No bootstrap required, opened a new log
I20260807 06:38:27.852632 18039 ts_tablet_manager.cc:1397] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:38:27.854218 18039 raft_consensus.cc:348] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.854341 18039 raft_consensus.cc:374] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:27.854360 18039 raft_consensus.cc:729] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76504f4822594df6a7d955e4fa06706d, State: Initialized, Role: FOLLOWER
I20260807 06:38:27.854466 18039 consensus_queue.cc:260] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.854692 18039 ts_tablet_manager.cc:1428] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:38:27.854755 17824 heartbeater.cc:503] Master 127.11.114.254:33505 was elected leader, sending a full tablet report...
W20260807 06:38:27.897517 17826 tablet.cc:2369] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:27.949517 18041 raft_consensus.cc:482] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:27.949673 18041 raft_consensus.cc:504] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.950035 18041 leader_election.cc:288] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6ec0d21b84244b78b8912cb7d43f7cb (127.11.114.193:42565), 76504f4822594df6a7d955e4fa06706d (127.11.114.194:36991)
I20260807 06:38:27.953440 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.950263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42565 (local address 127.11.114.195:34063)
0807 06:38:27.950327 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:27.950342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:27.950398 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:27.950556 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:27.950557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:27.950558 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:27.950620 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.950622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.951392 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.951393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.952232 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.952235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.953161 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.953162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.953252 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.953255 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:27.953325 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:38:27.953363 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 06:38:27.953440 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.950362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:34063 (local address 127.11.114.193:42565)
0807 06:38:27.950413 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.950415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.950427 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:38:27.950462 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.950463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.950474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.950548 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:27.950696 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.951300 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.952326 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.953022 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.953159 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.953171 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.953333 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.953373 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:38:27.953732 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.950383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:36605 (local address 127.11.114.194:36991)
0807 06:38:27.950429 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.950431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.950600 (+ 169us) server_negotiation.cc:407] Connection header received
0807 06:38:27.950743 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.950745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.950754 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.950796 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:27.950963 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.951486 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.952713 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.953314 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.953397 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.953406 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.953615 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.953668 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:38:27.953814 17578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23bccad48782400ab853c3f3ea09b47d" candidate_uuid: "3c2771a22d5049ca8792b780d78b8043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" is_pre_election: true
I20260807 06:38:27.953941 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.950295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:36991 (local address 127.11.114.195:36605)
0807 06:38:27.950514 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:27.950525 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:27.950632 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:27.950821 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:27.950822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:27.950824 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:27.950893 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.950894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.951623 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.951624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.952566 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:27.952586 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.953410 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.953411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:27.953481 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.953484 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:27.953814 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 06:38:27.953860 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":144,"threads_started":1}
I20260807 06:38:27.954002 17578 raft_consensus.cc:2454] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c2771a22d5049ca8792b780d78b8043 in term 0.
I20260807 06:38:27.954208 17856 leader_election.cc:302] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c2771a22d5049ca8792b780d78b8043, e6ec0d21b84244b78b8912cb7d43f7cb; no voters:
I20260807 06:38:27.954361 18041 raft_consensus.cc:2781] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:27.954429 18041 raft_consensus.cc:482] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:27.954377 17761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23bccad48782400ab853c3f3ea09b47d" candidate_uuid: "3c2771a22d5049ca8792b780d78b8043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76504f4822594df6a7d955e4fa06706d" is_pre_election: true
I20260807 06:38:27.954464 18041 raft_consensus.cc:3037] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:27.954545 17761 raft_consensus.cc:2454] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c2771a22d5049ca8792b780d78b8043 in term 0.
I20260807 06:38:27.955320 18041 raft_consensus.cc:504] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.955476 18041 leader_election.cc:288] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [CANDIDATE]: Term 1 election: Requested vote from peers e6ec0d21b84244b78b8912cb7d43f7cb (127.11.114.193:42565), 76504f4822594df6a7d955e4fa06706d (127.11.114.194:36991)
I20260807 06:38:27.955636 17761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23bccad48782400ab853c3f3ea09b47d" candidate_uuid: "3c2771a22d5049ca8792b780d78b8043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76504f4822594df6a7d955e4fa06706d"
I20260807 06:38:27.955651 17578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23bccad48782400ab853c3f3ea09b47d" candidate_uuid: "3c2771a22d5049ca8792b780d78b8043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb"
I20260807 06:38:27.955729 17578 raft_consensus.cc:3037] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:27.955732 17761 raft_consensus.cc:3037] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:27.956452 17761 raft_consensus.cc:2454] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c2771a22d5049ca8792b780d78b8043 in term 1.
I20260807 06:38:27.956489 17578 raft_consensus.cc:2454] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c2771a22d5049ca8792b780d78b8043 in term 1.
I20260807 06:38:27.956643 17857 leader_election.cc:302] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c2771a22d5049ca8792b780d78b8043, 76504f4822594df6a7d955e4fa06706d; no voters:
I20260807 06:38:27.956781 18041 raft_consensus.cc:2781] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:27.956931 18041 raft_consensus.cc:686] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 1 LEADER]: Becoming Leader. State: Replica: 3c2771a22d5049ca8792b780d78b8043, State: Running, Role: LEADER
I20260807 06:38:27.957046 18041 consensus_queue.cc:237] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } }
I20260807 06:38:27.957984 17381 catalog_manager.cc:5495] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c2771a22d5049ca8792b780d78b8043 (127.11.114.195). New cstate: current_term: 1 leader_uuid: "3c2771a22d5049ca8792b780d78b8043" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2771a22d5049ca8792b780d78b8043" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 44663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 } health_report { overall_health: UNKNOWN } } }
W20260807 06:38:27.977391 17640 tablet.cc:2369] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:27.986728 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:27.984203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49886 (local address 127.11.114.195:44663)
0807 06:38:27.984284 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:27.984286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:27.984294 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:27.984347 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:27.984348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:27.984368 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:27.984433 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:27.984584 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.985049 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.985879 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:27.986056 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:27.986107 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:27.986308 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:27.986408 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:27.986470 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:27.986581 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 06:38:27.986600 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 06:38:27.988838 17761 raft_consensus.cc:1261] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 1 FOLLOWER]: Refusing update from remote peer 3c2771a22d5049ca8792b780d78b8043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:27.988839 17578 raft_consensus.cc:1261] T 23bccad48782400ab853c3f3ea09b47d P e6ec0d21b84244b78b8912cb7d43f7cb [term 1 FOLLOWER]: Refusing update from remote peer 3c2771a22d5049ca8792b780d78b8043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:27.989219 18045 consensus_queue.cc:1035] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76504f4822594df6a7d955e4fa06706d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:27.989355 18041 consensus_queue.cc:1035] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ec0d21b84244b78b8912cb7d43f7cb" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:27.992987 18051 mvcc.cc:205] Tried to move back new op lower bound from 7315802963919208448 to 7315802963792871424. Current Snapshot: MvccSnapshot[applied={T|T < 7315802963919208448}]
I20260807 06:38:27.994091 17380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:27.994192 17380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:27.994985 17380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5882f7dafdc4a0187a6aa6b3314ab84: Sending DeleteTablet for 3 replicas of tablet 23bccad48782400ab853c3f3ea09b47d
I20260807 06:38:27.995325 17552 tablet_service.cc:1514] Processing DeleteTablet for tablet 23bccad48782400ab853c3f3ea09b47d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:27 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:35480
I20260807 06:38:27.995369 17918 tablet_service.cc:1514] Processing DeleteTablet for tablet 23bccad48782400ab853c3f3ea09b47d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:27 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:49872
I20260807 06:38:27.995587 17733 tablet_service.cc:1514] Processing DeleteTablet for tablet 23bccad48782400ab853c3f3ea09b47d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:27 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:33368
I20260807 06:38:27.995819 18060 tablet_replica.cc:330] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: stopping tablet replica
I20260807 06:38:27.995882 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 17466
I20260807 06:38:27.998158 18060 raft_consensus.cc:2229] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:27.998359 18060 raft_consensus.cc:2258] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:28.004134 18060 ts_tablet_manager.cc:1905] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:28.006620 18060 ts_tablet_manager.cc:1918] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:28.006709 18060 log.cc:1196] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/23bccad48782400ab853c3f3ea09b47d
I20260807 06:38:28.006774 18059 tablet_replica.cc:330] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: stopping tablet replica
I20260807 06:38:28.006927 18059 raft_consensus.cc:2229] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:28.007022 18060 ts_tablet_manager.cc:1939] T 23bccad48782400ab853c3f3ea09b47d P 76504f4822594df6a7d955e4fa06706d: Deleting consensus metadata
I20260807 06:38:28.007079 18059 raft_consensus.cc:2258] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:28.007555 17361 catalog_manager.cc:4841] TS 76504f4822594df6a7d955e4fa06706d (127.11.114.194:36991): tablet 23bccad48782400ab853c3f3ea09b47d (table test-table [id=8b63734f4cd74aa5a179e9bc3eeaa28a]) successfully deleted
I20260807 06:38:28.007838 18059 ts_tablet_manager.cc:1905] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:28.009234 18059 ts_tablet_manager.cc:1918] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:28.009339 18059 log.cc:1196] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/23bccad48782400ab853c3f3ea09b47d
I20260807 06:38:28.009771 18059 ts_tablet_manager.cc:1939] T 23bccad48782400ab853c3f3ea09b47d P 3c2771a22d5049ca8792b780d78b8043: Deleting consensus metadata
I20260807 06:38:28.010274 17361 catalog_manager.cc:4841] TS 3c2771a22d5049ca8792b780d78b8043 (127.11.114.195:44663): tablet 23bccad48782400ab853c3f3ea09b47d (table test-table [id=8b63734f4cd74aa5a179e9bc3eeaa28a]) successfully deleted
W20260807 06:38:28.012385 17359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:42565 (error 108)
I20260807 06:38:28.013010 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.012875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42565 (local address 127.0.0.1:35488)
0807 06:38:28.012933 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.012972 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:42565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 06:38:28.013078 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 17647
W20260807 06:38:28.013234 17359 catalog_manager.cc:4584] TS e6ec0d21b84244b78b8912cb7d43f7cb (127.11.114.193:42565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23bccad48782400ab853c3f3ea09b47d: Network error: Client connection negotiation failed: client connection to 127.11.114.193:42565: connect: Connection refused (error 111)
I20260807 06:38:28.034835 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 17832
I20260807 06:38:28.042862 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 17336
2026-08-07T06:38:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2258 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T06:38:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:28Z Disabled control of system clock
I20260807 06:38:28.080785 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39593
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:37933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:38:28.163251 18065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:28.163472 18065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:28.163496 18065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:28.164569 18065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:28.164606 18065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:28.164619 18065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:28.164631 18065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:28.164644 18065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:28.166069 18065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39593
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:28.166347 18065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:28.166662 18065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:28.171519 18071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:28.171660 18065 server_base.cc:1034] running on GCE node
W20260807 06:38:28.171593 18070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:28.171769 18073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:28.172009 18065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:28.172307 18065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:28.173514 18065 hybrid_clock.cc:648] HybridClock initialized: now 1786084708173499 us; error 33 us; skew 500 ppm
I20260807 06:38:28.175014 18065 webserver.cc:460] Webserver started at http://127.11.114.254:44615/ using document root <none> and password file <none>
I20260807 06:38:28.175329 18065 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:28.175391 18065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:28.175608 18065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:28.176880 18065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "42244bf4daf542239179842e218222a9"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "07cfff62b5a1d87eb9056fa0ad8d7ae7"
server_key_iv: "d010f0a643ce3d4ce238d7273b9fc254"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.177311 18065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "42244bf4daf542239179842e218222a9"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "07cfff62b5a1d87eb9056fa0ad8d7ae7"
server_key_iv: "d010f0a643ce3d4ce238d7273b9fc254"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.178932 18065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:28.179744 18079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.179919 18065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:28.180029 18065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "42244bf4daf542239179842e218222a9"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "07cfff62b5a1d87eb9056fa0ad8d7ae7"
server_key_iv: "d010f0a643ce3d4ce238d7273b9fc254"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.180137 18065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:28.183897 18065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:28.184170 18065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:28.184324 18065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:28.188416 18065 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39593
I20260807 06:38:28.188446 18160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39593 every 8 connection(s)
I20260807 06:38:28.188788 18065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:28.189455 18162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:28.192327 18162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9: Bootstrap starting.
I20260807 06:38:28.192904 18162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:28.193193 18162 log.cc:824] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:28.193873 18162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9: No bootstrap required, opened a new log
I20260807 06:38:28.195194 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 18065
I20260807 06:38:28.195351 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:28.195541 11723 external_mini_cluster.cc:1428] Setting key 2de5d5489f8bf254932f458a87a750cd
I20260807 06:38:28.195516 18162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42244bf4daf542239179842e218222a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39593 } }
I20260807 06:38:28.195677 18162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:28.195739 18162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42244bf4daf542239179842e218222a9, State: Initialized, Role: FOLLOWER
I20260807 06:38:28.195883 18162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42244bf4daf542239179842e218222a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39593 } }
I20260807 06:38:28.195971 18162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:28.196013 18162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:28.196058 18162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:28.196599 18162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42244bf4daf542239179842e218222a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39593 } }
I20260807 06:38:28.196718 18162 leader_election.cc:302] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42244bf4daf542239179842e218222a9; no voters:
I20260807 06:38:28.197140 18162 leader_election.cc:288] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:28.197192 18172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:28.197356 18172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [term 1 LEADER]: Becoming Leader. State: Replica: 42244bf4daf542239179842e218222a9, State: Running, Role: LEADER
I20260807 06:38:28.197458 18162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:28.198060 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.196064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52282 (local address 127.11.114.254:39593)
0807 06:38:28.196293 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.196302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.196338 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:38:28.196435 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.196440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.196473 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.196623 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.197271 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.197276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.197276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.197281 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.197354 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.197370 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.197416 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.197570 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.197636 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260807 06:38:28.197456 18172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42244bf4daf542239179842e218222a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39593 } }
I20260807 06:38:28.198891 18175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42244bf4daf542239179842e218222a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42244bf4daf542239179842e218222a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39593 } } }
I20260807 06:38:28.199034 18174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42244bf4daf542239179842e218222a9. Latest consensus state: current_term: 1 leader_uuid: "42244bf4daf542239179842e218222a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42244bf4daf542239179842e218222a9" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39593 } } }
I20260807 06:38:28.199321 18174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:28.199318 18175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9 [sys.catalog]: This master's current role is: LEADER
W20260807 06:38:28.199361 18192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:38:28.199458 18192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:28.199533 18195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:28.199717 18195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:28.201140 18195 catalog_manager.cc:1347] Generated new cluster ID: 76b3fae926624c829c28c4d808c66e40
I20260807 06:38:28.201196 18195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:28.217757 18195 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:28.217952 18195 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:28.222824 18195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9: Generated new TSK 0
I20260807 06:38:28.223016 18195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:38:28.226958 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39593
--builtin_ntp_servers=127.11.114.212:37933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:38:28.310853 18198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:28.311095 18198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:28.311132 18198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:28.312301 18198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:28.312366 18198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:28.312410 18198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:28.313803 18198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:28.314069 18198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:28.314373 18198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:28.320261 18205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:28.320318 18206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:28.320283 18209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:28.320549 18198 server_base.cc:1034] running on GCE node
I20260807 06:38:28.320825 18198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:28.321074 18198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:28.322237 18198 hybrid_clock.cc:648] HybridClock initialized: now 1786084708322210 us; error 44 us; skew 500 ppm
I20260807 06:38:28.323603 18198 webserver.cc:460] Webserver started at http://127.11.114.193:39645/ using document root <none> and password file <none>
I20260807 06:38:28.323856 18198 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:28.323928 18198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:28.324080 18198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:28.325169 18198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "b85f9fd339514fe79c33ccada5f9da31"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "8e1e2c1535441a0806619e664498d87a"
server_key_iv: "1fdd84addc1610b68e87bea2b735cf40"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.325636 18198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "b85f9fd339514fe79c33ccada5f9da31"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "8e1e2c1535441a0806619e664498d87a"
server_key_iv: "1fdd84addc1610b68e87bea2b735cf40"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.327041 18198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:28.327849 18218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.328099 18198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:28.328200 18198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "b85f9fd339514fe79c33ccada5f9da31"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "8e1e2c1535441a0806619e664498d87a"
server_key_iv: "1fdd84addc1610b68e87bea2b735cf40"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.328269 18198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:28.332940 18198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:28.333240 18198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:28.333365 18198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:28.333604 18198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:28.333936 18198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:28.333976 18198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.334003 18198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:28.334026 18198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.340413 18198 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:44921
I20260807 06:38:28.340493 18382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:44921 every 8 connection(s)
I20260807 06:38:28.340723 18198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:28.342602 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 18198
I20260807 06:38:28.342728 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:28.342939 11723 external_mini_cluster.cc:1428] Setting key a434063f1f6e30222c4bb44c6eb2f250
I20260807 06:38:28.344038 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.341438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:35181 (local address 127.11.114.254:39593)
0807 06:38:28.341525 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.341527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.341790 (+ 263us) server_negotiation.cc:407] Connection header received
0807 06:38:28.342278 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.342279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.342287 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.342354 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.343714 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.343715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.343716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.343722 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.343743 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.343747 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.343820 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.343924 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.343941 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:38:28.344147 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39593
--builtin_ntp_servers=127.11.114.212:37933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 06:38:28.344151 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.341407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39593 (local address 127.11.114.193:35181)
0807 06:38:28.341659 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.341692 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:28.342163 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:28.342424 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:28.342426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:28.342435 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:28.343521 (+ 1086us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:38:28.343522 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:28.343568 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:38:28.343602 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:28.343832 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:28.343838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:28.343840 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:38:28.343917 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:38:28.343968 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":125,"threads_started":1}
I20260807 06:38:28.344638 18383 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39593
I20260807 06:38:28.344729 18383 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:28.344893 18383 heartbeater.cc:511] Master 127.11.114.254:39593 requested a full tablet report, sending...
I20260807 06:38:28.345366 18110 ts_manager.cc:194] Registered new tserver with Master: b85f9fd339514fe79c33ccada5f9da31 (127.11.114.193:44921)
W20260807 06:38:28.428910 18386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:28.429140 18386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:28.429164 18386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:28.430418 18386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:28.430469 18386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:28.430496 18386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:28.432104 18386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:28.432385 18386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:28.432715 18386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:28.438117 18394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:28.438159 18397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:28.438143 18395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:28.439060 18386 server_base.cc:1034] running on GCE node
I20260807 06:38:28.439303 18386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:28.439581 18386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:28.440752 18386 hybrid_clock.cc:648] HybridClock initialized: now 1786084708440741 us; error 40 us; skew 500 ppm
I20260807 06:38:28.442139 18386 webserver.cc:460] Webserver started at http://127.11.114.194:44633/ using document root <none> and password file <none>
I20260807 06:38:28.442423 18386 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:28.442474 18386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:28.442649 18386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:28.443758 18386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "79bd2a3237984a338ee0c3fb6a0bf50b"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "9e220771bc841567be46ef70a6ba3bc8"
server_key_iv: "e95c6ac3b098f94241c39e054449d188"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.444113 18386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "79bd2a3237984a338ee0c3fb6a0bf50b"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "9e220771bc841567be46ef70a6ba3bc8"
server_key_iv: "e95c6ac3b098f94241c39e054449d188"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.445492 18386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:38:28.446285 18404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.446486 18386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:28.446561 18386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "79bd2a3237984a338ee0c3fb6a0bf50b"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "9e220771bc841567be46ef70a6ba3bc8"
server_key_iv: "e95c6ac3b098f94241c39e054449d188"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.446652 18386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:28.450189 18386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:28.450490 18386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:28.450632 18386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:28.450845 18386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:28.451153 18386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:28.451195 18386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.451244 18386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:28.451273 18386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.457247 18386 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:35617
I20260807 06:38:28.457309 18564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:35617 every 8 connection(s)
I20260807 06:38:28.457695 18386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:28.460145 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 18386
I20260807 06:38:28.460239 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:28.460472 11723 external_mini_cluster.cc:1428] Setting key b4082d5b96ae3f4d946cc55a8c9011e2
I20260807 06:38:28.461041 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.458503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:49609 (local address 127.11.114.254:39593)
0807 06:38:28.458570 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.458572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.458790 (+ 218us) server_negotiation.cc:407] Connection header received
0807 06:38:28.459387 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.459388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.459395 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.459455 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.460775 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.460777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.460778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.460784 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.460803 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.460809 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.460880 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.460976 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.460988 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:38:28.461289 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.458494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39593 (local address 127.11.114.194:49609)
0807 06:38:28.458671 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.458707 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:28.459267 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:28.459532 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:28.459536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:28.459547 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:28.460573 (+ 1026us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:38:28.460574 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:28.460616 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:38:28.460665 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:28.460893 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:28.460899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:28.460902 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:28.460988 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 06:38:28.461053 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260807 06:38:28.461594 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39593
--builtin_ntp_servers=127.11.114.212:37933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 06:38:28.461850 18565 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39593
I20260807 06:38:28.461964 18565 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:28.462121 18565 heartbeater.cc:511] Master 127.11.114.254:39593 requested a full tablet report, sending...
I20260807 06:38:28.462507 18110 ts_manager.cc:194] Registered new tserver with Master: 79bd2a3237984a338ee0c3fb6a0bf50b (127.11.114.194:35617)
W20260807 06:38:28.547771 18573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:28.548040 18573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:28.548084 18573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:28.549307 18573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:28.549357 18573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:28.549399 18573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:28.550829 18573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:28.551095 18573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:28.551462 18573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:28.556641 18579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:28.556677 18585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:28.556742 18573 server_base.cc:1034] running on GCE node
W20260807 06:38:28.556694 18580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:28.557047 18573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:28.557292 18573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:28.558450 18573 hybrid_clock.cc:648] HybridClock initialized: now 1786084708558442 us; error 37 us; skew 500 ppm
I20260807 06:38:28.559767 18573 webserver.cc:460] Webserver started at http://127.11.114.195:43015/ using document root <none> and password file <none>
I20260807 06:38:28.560030 18573 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:28.560102 18573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:28.560257 18573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:28.561321 18573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "723a4c50fc81431e80e70f624d59e2ef"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "750fa6efadfb3bf33f8cf39191099022"
server_key_iv: "d1f32b0d383aae6f80fb9a99ec450ff6"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.561697 18573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "723a4c50fc81431e80e70f624d59e2ef"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "750fa6efadfb3bf33f8cf39191099022"
server_key_iv: "d1f32b0d383aae6f80fb9a99ec450ff6"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.563004 18573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:28.563705 18594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.563869 18573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:28.563971 18573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "723a4c50fc81431e80e70f624d59e2ef"
format_stamp: "Formatted at 2026-08-07 06:38:28 on dist-test-slave-gkw7"
server_key: "750fa6efadfb3bf33f8cf39191099022"
server_key_iv: "d1f32b0d383aae6f80fb9a99ec450ff6"
server_key_version: "encryptionkey@0"
I20260807 06:38:28.564052 18573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:28.567476 18573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:28.567723 18573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:28.567862 18573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:28.568061 18573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:28.568403 18573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:28.568454 18573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.568495 18573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:28.568521 18573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:28.574419 18573 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:33027
I20260807 06:38:28.574465 18743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:33027 every 8 connection(s)
I20260807 06:38:28.574779 18573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:38:28.577064 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 18573
I20260807 06:38:28.577234 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:28.577454 11723 external_mini_cluster.cc:1428] Setting key 5f258cc587d111d915a6d9bbbb23ba08
I20260807 06:38:28.578585 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.575767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:53999 (local address 127.11.114.254:39593)
0807 06:38:28.575835 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.575837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.576108 (+ 271us) server_negotiation.cc:407] Connection header received
0807 06:38:28.576643 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.576644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.576651 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.576703 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.578333 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.578333 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.578334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.578339 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.578357 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.578362 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.578426 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.578518 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.578532 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:38:28.578787 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.575737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39593 (local address 127.11.114.195:53999)
0807 06:38:28.575954 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.575987 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:28.576510 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:28.576771 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:28.576775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:28.576788 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:28.578150 (+ 1362us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:38:28.578151 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:28.578189 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:38:28.578234 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:28.578435 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:28.578440 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:28.578443 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:28.578510 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:38:28.578560 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260807 06:38:28.579244 18746 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39593
I20260807 06:38:28.579339 18746 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:28.579485 18746 heartbeater.cc:511] Master 127.11.114.254:39593 requested a full tablet report, sending...
I20260807 06:38:28.579856 18110 ts_manager.cc:194] Registered new tserver with Master: 723a4c50fc81431e80e70f624d59e2ef (127.11.114.195:33027)
I20260807 06:38:28.589299 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:38:28.591949 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.590878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.11.114.254:39593)
0807 06:38:28.590936 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.590937 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.591052 (+ 115us) server_negotiation.cc:407] Connection header received
0807 06:38:28.591156 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.591157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.591163 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.591230 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.591772 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.591773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.591773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.591777 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.591793 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.591797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.591840 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.591879 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.591903 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:38:28.593497 18110 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:28.595216 18110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:28.599368 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.597715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.11.114.195:33027)
0807 06:38:28.598090 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.598104 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.598114 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:28.598138 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.598140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.598149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.598218 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.599021 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.599028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.599029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.599034 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.599072 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.599095 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.599160 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.599269 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.599285 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":69,"threads_started":1}
I20260807 06:38:28.599356 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.597553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44921 (local address 127.0.0.1:35950)
0807 06:38:28.597743 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.597783 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:28.597864 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:28.598261 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:28.598267 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:28.598273 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:28.598840 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:38:28.598841 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:28.598858 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:38:28.598880 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:28.599198 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:28.599199 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:28.599199 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 06:38:28.599248 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:38:28.599261 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260807 06:38:28.599443 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.597745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.11.114.193:44921)
0807 06:38:28.597926 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.597964 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.597977 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:38:28.598006 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.598008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.598019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.598184 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.599048 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.599054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.599054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.599058 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.599086 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.599117 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.599172 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.599270 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.599344 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260807 06:38:28.599560 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.597568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:33027 (local address 127.0.0.1:49196)
0807 06:38:28.597947 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.597958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:28.598003 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:28.598527 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:28.598529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:28.598532 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:28.598949 (+ 417us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:38:28.598950 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:28.598955 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:38:28.598964 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:28.599156 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:28.599163 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:28.599167 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:28.599262 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 06:38:28.599280 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":107,"threads_started":1}
I20260807 06:38:28.599939 18759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.597573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:35617 (local address 127.0.0.1:46108)
0807 06:38:28.597992 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.598001 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:28.598044 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:28.598790 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:28.598791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:28.598794 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:28.599404 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:38:28.599405 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:28.599411 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:38:28.599422 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:28.599805 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:28.599806 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:28.599807 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:38:28.599860 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 06:38:28.599874 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":94,"threads_started":1}
I20260807 06:38:28.599983 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.597804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.11.114.194:35617)
0807 06:38:28.598011 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.598015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.598114 (+ 99us) server_negotiation.cc:407] Connection header received
0807 06:38:28.598147 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.598150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.598161 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.598382 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.599518 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.599527 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.599527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.599532 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.599596 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.599721 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.599790 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.599880 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.599898 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
I20260807 06:38:28.600215 18285 tablet_service.cc:1467] Processing CreateTablet for tablet e244ce44118e4357b1b44474ba06ea81 (DEFAULT_TABLE table=test-table [id=853245da095d4e29af93e48824de345c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:28.600569 18285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e244ce44118e4357b1b44474ba06ea81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:28.600468 18660 tablet_service.cc:1467] Processing CreateTablet for tablet e244ce44118e4357b1b44474ba06ea81 (DEFAULT_TABLE table=test-table [id=853245da095d4e29af93e48824de345c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:28.600585 18471 tablet_service.cc:1467] Processing CreateTablet for tablet e244ce44118e4357b1b44474ba06ea81 (DEFAULT_TABLE table=test-table [id=853245da095d4e29af93e48824de345c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:28.600818 18660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e244ce44118e4357b1b44474ba06ea81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:28.600883 18471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e244ce44118e4357b1b44474ba06ea81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:28.603518 18766 tablet_bootstrap.cc:492] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31: Bootstrap starting.
I20260807 06:38:28.604183 18768 tablet_bootstrap.cc:492] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Bootstrap starting.
I20260807 06:38:28.604465 18766 tablet_bootstrap.cc:654] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:28.604861 18766 log.cc:824] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:28.604899 18768 tablet_bootstrap.cc:654] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:28.605244 18768 log.cc:824] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:28.605659 18767 tablet_bootstrap.cc:492] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Bootstrap starting.
I20260807 06:38:28.605705 18766 tablet_bootstrap.cc:492] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31: No bootstrap required, opened a new log
I20260807 06:38:28.605794 18766 ts_tablet_manager.cc:1397] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:38:28.606180 18768 tablet_bootstrap.cc:492] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: No bootstrap required, opened a new log
I20260807 06:38:28.606247 18768 ts_tablet_manager.cc:1397] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:28.606415 18767 tablet_bootstrap.cc:654] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:28.606734 18767 log.cc:824] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:28.607482 18767 tablet_bootstrap.cc:492] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: No bootstrap required, opened a new log
I20260807 06:38:28.607573 18767 ts_tablet_manager.cc:1397] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:28.608546 18766 raft_consensus.cc:348] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.608728 18766 raft_consensus.cc:374] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:28.608767 18766 raft_consensus.cc:729] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85f9fd339514fe79c33ccada5f9da31, State: Initialized, Role: FOLLOWER
I20260807 06:38:28.608913 18766 consensus_queue.cc:260] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.608881 18768 raft_consensus.cc:348] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.609014 18768 raft_consensus.cc:374] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:28.609056 18768 raft_consensus.cc:729] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 723a4c50fc81431e80e70f624d59e2ef, State: Initialized, Role: FOLLOWER
I20260807 06:38:28.609212 18766 ts_tablet_manager.cc:1428] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 06:38:28.609202 18768 consensus_queue.cc:260] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.609409 18383 heartbeater.cc:503] Master 127.11.114.254:39593 was elected leader, sending a full tablet report...
I20260807 06:38:28.609503 18768 ts_tablet_manager.cc:1428] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:38:28.609560 18746 heartbeater.cc:503] Master 127.11.114.254:39593 was elected leader, sending a full tablet report...
I20260807 06:38:28.610438 18767 raft_consensus.cc:348] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.610622 18767 raft_consensus.cc:374] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:28.610666 18767 raft_consensus.cc:729] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bd2a3237984a338ee0c3fb6a0bf50b, State: Initialized, Role: FOLLOWER
I20260807 06:38:28.610793 18767 consensus_queue.cc:260] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.611140 18767 ts_tablet_manager.cc:1428] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 06:38:28.611289 18565 heartbeater.cc:503] Master 127.11.114.254:39593 was elected leader, sending a full tablet report...
W20260807 06:38:28.708386 18568 tablet.cc:2369] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:38:28.825340 18748 tablet.cc:2369] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:38:28.841377 18384 tablet.cc:2369] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:28.902014 18779 raft_consensus.cc:482] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:28.902150 18779 raft_consensus.cc:504] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.902527 18779 leader_election.cc:288] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b85f9fd339514fe79c33ccada5f9da31 (127.11.114.193:44921), 723a4c50fc81431e80e70f624d59e2ef (127.11.114.195:33027)
I20260807 06:38:28.904178 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.902789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:33027 (local address 127.11.114.194:52649)
0807 06:38:28.902850 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.902862 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:28.902906 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:28.903135 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:28.903136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:28.903140 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:28.903701 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:38:28.903702 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:28.903709 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:38:28.903720 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:28.904024 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:28.904025 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:28.904026 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:38:28.904083 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:38:28.904098 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 06:38:28.904212 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.902884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:52649 (local address 127.11.114.195:33027)
0807 06:38:28.902942 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.902945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.902958 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:38:28.902992 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.902993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.903000 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.903122 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.903886 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.903888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.903888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.903894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.903939 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.903946 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.904041 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.904106 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.904124 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 06:38:28.904232 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.902900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:50849 (local address 127.11.114.193:44921)
0807 06:38:28.902947 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.902949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.903134 (+ 185us) server_negotiation.cc:407] Connection header received
0807 06:38:28.903176 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.903177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.903185 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.903256 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.903970 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.903972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.903972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.903978 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.903998 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.904004 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.904079 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.904144 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.904160 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:28.904208 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.902802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44921 (local address 127.11.114.194:50849)
0807 06:38:28.903028 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.903035 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:28.903103 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:28.903284 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:28.903286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:28.903288 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:28.903852 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:38:28.903853 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:28.903858 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:38:28.903867 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:28.904079 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:28.904080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:28.904081 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:38:28.904125 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 06:38:28.904137 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":127,"threads_started":1}
I20260807 06:38:28.904660 18315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e244ce44118e4357b1b44474ba06ea81" candidate_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b85f9fd339514fe79c33ccada5f9da31" is_pre_election: true
I20260807 06:38:28.904711 18685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e244ce44118e4357b1b44474ba06ea81" candidate_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723a4c50fc81431e80e70f624d59e2ef" is_pre_election: true
I20260807 06:38:28.904832 18315 raft_consensus.cc:2454] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79bd2a3237984a338ee0c3fb6a0bf50b in term 0.
I20260807 06:38:28.904887 18685 raft_consensus.cc:2454] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79bd2a3237984a338ee0c3fb6a0bf50b in term 0.
I20260807 06:38:28.905098 18411 leader_election.cc:302] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 723a4c50fc81431e80e70f624d59e2ef, 79bd2a3237984a338ee0c3fb6a0bf50b; no voters:
I20260807 06:38:28.905241 18779 raft_consensus.cc:2781] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:28.905297 18779 raft_consensus.cc:482] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:28.905320 18779 raft_consensus.cc:3037] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:28.906112 18779 raft_consensus.cc:504] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.906252 18779 leader_election.cc:288] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [CANDIDATE]: Term 1 election: Requested vote from peers b85f9fd339514fe79c33ccada5f9da31 (127.11.114.193:44921), 723a4c50fc81431e80e70f624d59e2ef (127.11.114.195:33027)
I20260807 06:38:28.906448 18685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e244ce44118e4357b1b44474ba06ea81" candidate_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723a4c50fc81431e80e70f624d59e2ef"
I20260807 06:38:28.906544 18685 raft_consensus.cc:3037] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:28.907325 18685 raft_consensus.cc:2454] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79bd2a3237984a338ee0c3fb6a0bf50b in term 1.
I20260807 06:38:28.907486 18315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e244ce44118e4357b1b44474ba06ea81" candidate_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b85f9fd339514fe79c33ccada5f9da31"
I20260807 06:38:28.907519 18411 leader_election.cc:302] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 723a4c50fc81431e80e70f624d59e2ef, 79bd2a3237984a338ee0c3fb6a0bf50b; no voters:
I20260807 06:38:28.907596 18779 raft_consensus.cc:2781] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:28.907580 18315 raft_consensus.cc:3037] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:28.907752 18779 raft_consensus.cc:686] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 1 LEADER]: Becoming Leader. State: Replica: 79bd2a3237984a338ee0c3fb6a0bf50b, State: Running, Role: LEADER
I20260807 06:38:28.907967 18779 consensus_queue.cc:237] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } }
I20260807 06:38:28.908497 18315 raft_consensus.cc:2454] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79bd2a3237984a338ee0c3fb6a0bf50b in term 1.
I20260807 06:38:28.908859 18109 catalog_manager.cc:5495] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b reported cstate change: term changed from 0 to 1, leader changed from <none> to 79bd2a3237984a338ee0c3fb6a0bf50b (127.11.114.194). New cstate: current_term: 1 leader_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79bd2a3237984a338ee0c3fb6a0bf50b" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 35617 } health_report { overall_health: HEALTHY } } }
I20260807 06:38:28.956993 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.955937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.11.114.194:35617)
0807 06:38:28.956006 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:28.956008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:28.956032 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:38:28.956072 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:28.956072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:28.956085 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:28.956131 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:28.956778 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:28.956779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:28.956779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:28.956782 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:28.956793 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:28.956797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:28.956838 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:28.956882 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 06:38:28.956893 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:38:28.959067 18315 raft_consensus.cc:1261] T e244ce44118e4357b1b44474ba06ea81 P b85f9fd339514fe79c33ccada5f9da31 [term 1 FOLLOWER]: Refusing update from remote peer 79bd2a3237984a338ee0c3fb6a0bf50b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:28.959085 18685 raft_consensus.cc:1261] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 1 FOLLOWER]: Refusing update from remote peer 79bd2a3237984a338ee0c3fb6a0bf50b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:28.959508 18781 consensus_queue.cc:1035] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85f9fd339514fe79c33ccada5f9da31" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:28.959648 18779 consensus_queue.cc:1035] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [LEADER]: Connected to new peer: Peer: permanent_uuid: "723a4c50fc81431e80e70f624d59e2ef" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 33027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:28.962189 18786 mvcc.cc:205] Tried to move back new op lower bound from 7315802967893168128 to 7315802967687708672. Current Snapshot: MvccSnapshot[applied={T|T < 7315802967893168128}]
I20260807 06:38:28.962908 18789 mvcc.cc:205] Tried to move back new op lower bound from 7315802967893168128 to 7315802967687708672. Current Snapshot: MvccSnapshot[applied={T|T < 7315802967893168128}]
I20260807 06:38:28.964437 18110 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:28.964520 18110 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:28.965313 18110 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42244bf4daf542239179842e218222a9: Sending DeleteTablet for 3 replicas of tablet e244ce44118e4357b1b44474ba06ea81
I20260807 06:38:28.965636 18285 tablet_service.cc:1514] Processing DeleteTablet for tablet e244ce44118e4357b1b44474ba06ea81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:28 UTC) from {username='slave'} at 127.0.0.1:35950
I20260807 06:38:28.965659 18660 tablet_service.cc:1514] Processing DeleteTablet for tablet e244ce44118e4357b1b44474ba06ea81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:28 UTC) from {username='slave'} at 127.0.0.1:49196
I20260807 06:38:28.966005 18800 tablet_replica.cc:330] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: stopping tablet replica
I20260807 06:38:28.965981 18471 tablet_service.cc:1514] Processing DeleteTablet for tablet e244ce44118e4357b1b44474ba06ea81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:28 UTC) from {username='slave'} at 127.0.0.1:46108
I20260807 06:38:28.966013 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 18198
I20260807 06:38:28.966104 18800 raft_consensus.cc:2229] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:28.966224 18800 raft_consensus.cc:2258] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:28.966624 18800 ts_tablet_manager.cc:1905] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:28.966898 18803 tablet_replica.cc:330] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: stopping tablet replica
I20260807 06:38:28.966976 18803 raft_consensus.cc:2229] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:28.967053 18803 raft_consensus.cc:2258] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:28.968923 18803 ts_tablet_manager.cc:1905] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:28.969877 18800 ts_tablet_manager.cc:1918] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:28.969944 18800 log.cc:1196] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/e244ce44118e4357b1b44474ba06ea81
I20260807 06:38:28.970201 18800 ts_tablet_manager.cc:1939] T e244ce44118e4357b1b44474ba06ea81 P 723a4c50fc81431e80e70f624d59e2ef: Deleting consensus metadata
I20260807 06:38:28.970604 18803 ts_tablet_manager.cc:1918] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:28.970597 18087 catalog_manager.cc:4841] TS 723a4c50fc81431e80e70f624d59e2ef (127.11.114.195:33027): tablet e244ce44118e4357b1b44474ba06ea81 (table test-table [id=853245da095d4e29af93e48824de345c]) successfully deleted
I20260807 06:38:28.970666 18803 log.cc:1196] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/e244ce44118e4357b1b44474ba06ea81
I20260807 06:38:28.970919 18803 ts_tablet_manager.cc:1939] T e244ce44118e4357b1b44474ba06ea81 P 79bd2a3237984a338ee0c3fb6a0bf50b: Deleting consensus metadata
I20260807 06:38:28.971375 18085 catalog_manager.cc:4841] TS 79bd2a3237984a338ee0c3fb6a0bf50b (127.11.114.194:35617): tablet e244ce44118e4357b1b44474ba06ea81 (table test-table [id=853245da095d4e29af93e48824de345c]) successfully deleted
W20260807 06:38:28.979852 18080 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:44921 (error 108)
I20260807 06:38:28.980193 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 18386
I20260807 06:38:28.980366 18759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:28.980230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44921 (local address 127.0.0.1:35958)
0807 06:38:28.980284 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:28.980322 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:38:28.988157 18080 catalog_manager.cc:4584] TS b85f9fd339514fe79c33ccada5f9da31 (127.11.114.193:44921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e244ce44118e4357b1b44474ba06ea81: Network error: Client connection negotiation failed: client connection to 127.11.114.193:44921: connect: Connection refused (error 111)
I20260807 06:38:28.992995 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 18573
I20260807 06:38:29.000394 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 18065
2026-08-07T06:38:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (952 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 06:38:29.024838 11723 mini_oidc.cc:152] Starting JWKS server
I20260807 06:38:29.026185 11723 webserver.cc:460] Webserver started at https://127.0.0.1:34189/ using document root <none> and password file <none>
I20260807 06:38:29.026326 11723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:38:29.026520 11723 webserver.cc:460] Webserver started at http://127.0.0.1:33567/ using document root <none> and password file <none>
2026-08-07T06:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:29Z Disabled control of system clock
I20260807 06:38:29.035600 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:35961
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:42865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:35961
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34189/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/testchainca.cert with env {}
W20260807 06:38:29.117975 18813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:29.118196 18813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:29.118220 18813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:29.119520 18813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:29.119575 18813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:29.119587 18813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:29.119601 18813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:29.119612 18813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:38:29.119625 18813 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:38:29.119637 18813 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34189/jwks.json
I20260807 06:38:29.121346 18813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:35961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:35961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34189/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:29.121650 18813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:29.121909 18813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:29.126900 18822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:29.126879 18824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:29.127099 18813 server_base.cc:1034] running on GCE node
W20260807 06:38:29.126881 18820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:29.127523 18813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:29.127871 18813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:29.129041 18813 hybrid_clock.cc:648] HybridClock initialized: now 1786084709129026 us; error 28 us; skew 500 ppm
I20260807 06:38:29.130330 18813 webserver.cc:460] Webserver started at http://127.11.114.254:40261/ using document root <none> and password file <none>
I20260807 06:38:29.130642 18813 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:29.130713 18813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:29.130867 18813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:29.132287 18813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "ae16e6e4986e4e96890049494e1f1788"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "c9157e153433b0b90c3c055acffb47ae"
server_key_iv: "3b702b4337d2e8bc805756be0ad6c7e1"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.132601 18813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "ae16e6e4986e4e96890049494e1f1788"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "c9157e153433b0b90c3c055acffb47ae"
server_key_iv: "3b702b4337d2e8bc805756be0ad6c7e1"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.133891 18813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 06:38:29.134570 18833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:29.134742 18813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:29.134843 18813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "ae16e6e4986e4e96890049494e1f1788"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "c9157e153433b0b90c3c055acffb47ae"
server_key_iv: "3b702b4337d2e8bc805756be0ad6c7e1"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.134927 18813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:29.149212 18813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:29.149641 18813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:29.149804 18813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:29.153733 18813 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:35961
I20260807 06:38:29.153775 18914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:35961 every 8 connection(s)
I20260807 06:38:29.154130 18813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:29.155189 18916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:29.157582 18916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788: Bootstrap starting.
I20260807 06:38:29.158187 18916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:29.158493 18916 log.cc:824] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:29.159198 18916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788: No bootstrap required, opened a new log
I20260807 06:38:29.160126 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 18813
I20260807 06:38:29.160266 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:29.160459 11723 external_mini_cluster.cc:1428] Setting key e33f543f1e199a9326162f70e5d16d84
I20260807 06:38:29.161024 18916 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae16e6e4986e4e96890049494e1f1788" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35961 } }
I20260807 06:38:29.161211 18916 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:29.161271 18916 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae16e6e4986e4e96890049494e1f1788, State: Initialized, Role: FOLLOWER
I20260807 06:38:29.161403 18916 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae16e6e4986e4e96890049494e1f1788" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35961 } }
I20260807 06:38:29.161477 18916 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:29.161499 18916 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:29.161595 18916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:29.162142 18916 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae16e6e4986e4e96890049494e1f1788" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35961 } }
I20260807 06:38:29.162263 18916 leader_election.cc:302] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae16e6e4986e4e96890049494e1f1788; no voters:
I20260807 06:38:29.162474 18916 leader_election.cc:288] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:29.162510 18925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:29.162621 18925 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [term 1 LEADER]: Becoming Leader. State: Replica: ae16e6e4986e4e96890049494e1f1788, State: Running, Role: LEADER
I20260807 06:38:29.162760 18916 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:29.162766 18925 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae16e6e4986e4e96890049494e1f1788" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35961 } }
I20260807 06:38:29.163213 18925 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae16e6e4986e4e96890049494e1f1788. Latest consensus state: current_term: 1 leader_uuid: "ae16e6e4986e4e96890049494e1f1788" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae16e6e4986e4e96890049494e1f1788" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35961 } } }
I20260807 06:38:29.163496 18925 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:29.163414 18928 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae16e6e4986e4e96890049494e1f1788" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae16e6e4986e4e96890049494e1f1788" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35961 } } }
I20260807 06:38:29.163908 18937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:29.163916 18928 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:29.164451 18937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:29.166484 18937 catalog_manager.cc:1347] Generated new cluster ID: 00312f83b09048ba95995b2d1e1963a4
I20260807 06:38:29.166548 18937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:29.167011 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:29.161017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.11.114.254:35961)
0807 06:38:29.161237 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:29.161247 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:29.161279 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:38:29.161407 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:29.161412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:29.161465 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:29.161649 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:29.161885 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.163485 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.164615 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.165242 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.165371 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.166199 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:29.166202 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:29.166203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:29.166407 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:29.166449 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:29.166465 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:29.166528 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:29.166750 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 06:38:29.166823 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260807 06:38:29.168005 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:38:29.176543 18937 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:29.177093 18937 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:29.185515 18937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae16e6e4986e4e96890049494e1f1788: Generated new TSK 0
I20260807 06:38:29.185731 18937 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 06:38:29.216471 18962 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 06:38:29.216635 18962 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260807 06:38:29.217284 18961 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:29.198976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35961 (local address 127.0.0.1:37564)
0807 06:38:29.199168 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:29.199179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:29.199238 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:29.199415 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:29.199416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:29.199423 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:38:29.199483 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:29.199485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.200456 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.200457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:29.201103 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:29.201106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.201659 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.201661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:29.201722 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.201743 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:38:29.216770 (+ 15027us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 06:38:29.216877 (+ 107us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.114.254:35961: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260807 06:38:29.217614 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:29.199088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.11.114.254:35961)
0807 06:38:29.199162 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:29.199165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:29.199239 (+ 74us) server_negotiation.cc:407] Connection header received
0807 06:38:29.199314 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:29.199315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:29.199328 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:29.199404 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:38:29.199548 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.200348 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.201197 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.201524 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.201615 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.201883 (+ 268us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:38:29.216595 (+ 14712us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 06:38:29.217549 (+ 954us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37564: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":46}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.114.254:35961: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 06:38:29.217691 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 18813
2026-08-07T06:38:29Z chronyd exiting
I20260807 06:38:29.264278 11723 test_util.cc:175] -----------------------------------------------
I20260807 06:38:29.264384 11723 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084684348861-11723-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 06:38:29.265246 11723 mini_oidc.cc:152] Starting JWKS server
I20260807 06:38:29.266031 11723 webserver.cc:460] Webserver started at https://127.0.0.1:41321/ using document root <none> and password file <none>
I20260807 06:38:29.266101 11723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:38:29.266299 11723 webserver.cc:460] Webserver started at http://127.0.0.1:34881/ using document root <none> and password file <none>
2026-08-07T06:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:29Z Disabled control of system clock
I20260807 06:38:29.275341 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:36569
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:36569
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41321/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/testchainca.cert with env {}
W20260807 06:38:29.364662 18970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:29.364923 18970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:29.364961 18970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:29.366222 18970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:29.366284 18970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:29.366307 18970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:29.366328 18970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:29.366356 18970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:38:29.366382 18970 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:38:29.366412 18970 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41321/jwks.json
I20260807 06:38:29.367933 18970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:36569
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:36569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41321/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:29.368204 18970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:29.368429 18970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:29.373430 18978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:29.373411 18982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:29.373397 18980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:29.373812 18970 server_base.cc:1034] running on GCE node
I20260807 06:38:29.374063 18970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:29.374368 18970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:29.375545 18970 hybrid_clock.cc:648] HybridClock initialized: now 1786084709375536 us; error 35 us; skew 500 ppm
I20260807 06:38:29.376859 18970 webserver.cc:460] Webserver started at http://127.11.114.254:42335/ using document root <none> and password file <none>
I20260807 06:38:29.377153 18970 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:29.377238 18970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:29.377390 18970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:29.378633 18970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "25d179b4fd7772678be2038392d0b85e"
server_key_iv: "0a3a706634eb79a604ea88e22e5ad984"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.379045 18970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "25d179b4fd7772678be2038392d0b85e"
server_key_iv: "0a3a706634eb79a604ea88e22e5ad984"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.380472 18970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:38:29.381158 18991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:29.381351 18970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:29.381461 18970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "25d179b4fd7772678be2038392d0b85e"
server_key_iv: "0a3a706634eb79a604ea88e22e5ad984"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.381556 18970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:29.407662 18970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:29.408028 18970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:29.408210 18970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:29.412519 18970 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:36569
I20260807 06:38:29.412575 19067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:36569 every 8 connection(s)
I20260807 06:38:29.413021 18970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:29.413796 19069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:29.416754 19069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31: Bootstrap starting.
I20260807 06:38:29.416872 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 18970
I20260807 06:38:29.416968 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:29.417171 11723 external_mini_cluster.cc:1428] Setting key 0ffb539ed75d584da1c829a9b8fa9274
I20260807 06:38:29.417639 19069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:29.418000 19069 log.cc:824] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:29.418834 19069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31: No bootstrap required, opened a new log
I20260807 06:38:29.420864 19069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36569 } }
I20260807 06:38:29.421046 19069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:29.421083 19069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ea0e48dc2c4897a2b44cd7c85a7e31, State: Initialized, Role: FOLLOWER
I20260807 06:38:29.421213 19069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36569 } }
I20260807 06:38:29.421293 19069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:29.421316 19069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:29.421384 19069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:29.422137 19069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36569 } }
I20260807 06:38:29.422228 19069 leader_election.cc:302] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29ea0e48dc2c4897a2b44cd7c85a7e31; no voters:
I20260807 06:38:29.422415 19069 leader_election.cc:288] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:29.422472 19077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:29.422636 19077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [term 1 LEADER]: Becoming Leader. State: Replica: 29ea0e48dc2c4897a2b44cd7c85a7e31, State: Running, Role: LEADER
I20260807 06:38:29.422693 19069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:29.422734 19077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36569 } }
I20260807 06:38:29.423046 19077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29ea0e48dc2c4897a2b44cd7c85a7e31. Latest consensus state: current_term: 1 leader_uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36569 } } }
I20260807 06:38:29.423100 19077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:29.423521 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:29.417738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39640 (local address 127.11.114.254:36569)
0807 06:38:29.417953 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:29.417961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:29.417986 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:38:29.418481 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:29.418488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:29.418550 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:29.418672 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:29.418898 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.419861 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.421024 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.421712 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.421814 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.422537 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:29.422541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:29.422542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:29.422752 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:29.422798 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:29.422812 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:29.423136 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:29.423171 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 06:38:29.423281 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260807 06:38:29.424163 19081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ea0e48dc2c4897a2b44cd7c85a7e31" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36569 } } }
I20260807 06:38:29.424396 19081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:29.424431 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 06:38:29.424587 19099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:38:29.424654 19099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:29.425182 19086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:29.425377 19086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:29.426772 19086 catalog_manager.cc:1347] Generated new cluster ID: ecad77e607cb4c80beced4589f39fb3e
I20260807 06:38:29.426823 19086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:29.431830 19086 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:29.432345 19086 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:29.445461 19086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29ea0e48dc2c4897a2b44cd7c85a7e31: Generated new TSK 0
I20260807 06:38:29.445750 19086 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 06:38:29.481303 19114 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 06:38:29.481460 19114 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 06:38:29.481858 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:29.458652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.11.114.254:36569)
0807 06:38:29.458699 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:29.458702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:29.458768 (+ 66us) server_negotiation.cc:407] Connection header received
0807 06:38:29.458849 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:29.458850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:29.458863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:29.458950 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:38:29.459114 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.460003 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.461165 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.461589 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.461675 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.461875 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:38:29.481462 (+ 19587us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41321/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 06:38:29.481745 (+ 283us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39656: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41321/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 06:38:29.481941 19074 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39656: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41321/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 06:38:29.481959 19111 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:29.458568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:36569 (local address 127.0.0.1:39656)
0807 06:38:29.458713 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:29.458723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:29.458782 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:29.458959 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:29.458960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:29.458963 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:38:29.459024 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:29.459027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.460097 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.460098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:29.461075 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:29.461078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.461677 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.461679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:29.461745 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.461751 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:38:29.481689 (+ 19938us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41321/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 06:38:29.481837 (+ 148us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.114.254:36569: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41321/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260807 06:38:29.482262 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 18970
2026-08-07T06:38:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (259 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 06:38:29.524821 11723 mini_oidc.cc:152] Starting JWKS server
I20260807 06:38:29.525612 11723 webserver.cc:460] Webserver started at https://127.0.0.1:38561/ using document root <none> and password file <none>
I20260807 06:38:29.525671 11723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:38:29.525868 11723 webserver.cc:460] Webserver started at http://127.0.0.1:33161/ using document root <none> and password file <none>
2026-08-07T06:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:29Z Disabled control of system clock
I20260807 06:38:29.535127 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39125
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:32989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39125
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38561/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 06:38:29.620550 19123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:29.620764 19123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:29.620787 19123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:29.621995 19123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:29.622046 19123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:29.622059 19123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:29.622072 19123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:29.622120 19123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:38:29.622146 19123 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:38:29.622166 19123 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38561/jwks.json
I20260807 06:38:29.623577 19123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:32989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38561/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:29.623845 19123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:29.624159 19123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:29.629333 19132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:29.629374 19130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:29.629407 19129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:29.629689 19123 server_base.cc:1034] running on GCE node
I20260807 06:38:29.629952 19123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:29.630229 19123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:29.631399 19123 hybrid_clock.cc:648] HybridClock initialized: now 1786084709631386 us; error 34 us; skew 500 ppm
I20260807 06:38:29.632829 19123 webserver.cc:460] Webserver started at http://127.11.114.254:40805/ using document root <none> and password file <none>
I20260807 06:38:29.633121 19123 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:29.633173 19123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:29.633363 19123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:29.634518 19123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "644f4b956fb14bcc803085dfdcfcb575"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "eb04b77d5b045135fb54e4b8e9b9e8a0"
server_key_iv: "87470cab13222ca0f8f42842997244f3"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.634963 19123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "644f4b956fb14bcc803085dfdcfcb575"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "eb04b77d5b045135fb54e4b8e9b9e8a0"
server_key_iv: "87470cab13222ca0f8f42842997244f3"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.636432 19123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:38:29.637202 19142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:29.637382 19123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:29.637490 19123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "644f4b956fb14bcc803085dfdcfcb575"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "eb04b77d5b045135fb54e4b8e9b9e8a0"
server_key_iv: "87470cab13222ca0f8f42842997244f3"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.637598 19123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:29.671252 19123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:29.671655 19123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:29.671844 19123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:29.676412 19123 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39125
I20260807 06:38:29.676473 19214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39125 every 8 connection(s)
I20260807 06:38:29.676870 19123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:29.677626 19215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:29.680296 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 19123
I20260807 06:38:29.680406 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:29.680531 19215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575: Bootstrap starting.
I20260807 06:38:29.680604 11723 external_mini_cluster.cc:1428] Setting key c12e9d57712e7b1fd17ece92c393c28a
I20260807 06:38:29.681293 19215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:29.681663 19215 log.cc:824] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:29.682485 19215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575: No bootstrap required, opened a new log
I20260807 06:38:29.684237 19215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644f4b956fb14bcc803085dfdcfcb575" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39125 } }
I20260807 06:38:29.684367 19215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:29.684389 19215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 644f4b956fb14bcc803085dfdcfcb575, State: Initialized, Role: FOLLOWER
I20260807 06:38:29.684525 19215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644f4b956fb14bcc803085dfdcfcb575" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39125 } }
I20260807 06:38:29.684602 19215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:29.684623 19215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:29.684691 19215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:29.685379 19215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644f4b956fb14bcc803085dfdcfcb575" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39125 } }
I20260807 06:38:29.685494 19215 leader_election.cc:302] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 644f4b956fb14bcc803085dfdcfcb575; no voters:
I20260807 06:38:29.685694 19215 leader_election.cc:288] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:29.685760 19222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:29.685935 19222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [term 1 LEADER]: Becoming Leader. State: Replica: 644f4b956fb14bcc803085dfdcfcb575, State: Running, Role: LEADER
I20260807 06:38:29.685945 19215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:29.686071 19222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644f4b956fb14bcc803085dfdcfcb575" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39125 } }
I20260807 06:38:29.686457 19222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 644f4b956fb14bcc803085dfdcfcb575. Latest consensus state: current_term: 1 leader_uuid: "644f4b956fb14bcc803085dfdcfcb575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644f4b956fb14bcc803085dfdcfcb575" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39125 } } }
I20260807 06:38:29.686474 19223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "644f4b956fb14bcc803085dfdcfcb575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644f4b956fb14bcc803085dfdcfcb575" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39125 } } }
I20260807 06:38:29.686563 19222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:29.686573 19223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:29.686805 19233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:29.687352 19233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:29.687448 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:29.681235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.11.114.254:39125)
0807 06:38:29.681524 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:29.681555 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:29.681593 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:38:29.682175 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:29.682184 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:29.682264 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:29.682419 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:29.682682 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.683706 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.684753 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.685387 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.685556 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.686442 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:29.686447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:29.686447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:29.686674 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:29.686719 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:29.686751 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:29.686819 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:29.687090 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 06:38:29.687200 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":109,"threads_started":1}
I20260807 06:38:29.688442 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:38:29.689203 19233 catalog_manager.cc:1347] Generated new cluster ID: 0e59da95e986440ca2c97a76cf55f00a
I20260807 06:38:29.689267 19233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:29.701351 19233 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:29.701963 19233 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:29.712052 19233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 644f4b956fb14bcc803085dfdcfcb575: Generated new TSK 0
I20260807 06:38:29.712293 19233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:38:29.727264 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:29.723594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.11.114.254:39125)
0807 06:38:29.723668 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:29.723671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:29.723782 (+ 111us) server_negotiation.cc:407] Connection header received
0807 06:38:29.723865 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:29.723866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:29.723880 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:29.723973 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:38:29.724119 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.724636 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.725445 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.725756 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.725848 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.726064 (+ 216us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:38:29.727046 (+ 982us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 06:38:29.727188 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40982: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 06:38:29.727339 19219 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40982: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 06:38:29.727468 19253 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:29.723517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39125 (local address 127.0.0.1:40982)
0807 06:38:29.723711 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:29.723721 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:29.723800 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:29.723970 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:29.723971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:29.723973 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:38:29.724037 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:29.724040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.724727 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.724728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:29.725352 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:29.725354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.725888 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.725890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:29.725948 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.725953 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:38:29.727152 (+ 1199us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 06:38:29.727363 (+ 211us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.114.254:39125: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":116,"threads_started":1}
I20260807 06:38:29.727828 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 19123
E20260807 06:38:29.744578 19254 webserver.cc:533] Webserver: sslize failed:
E20260807 06:38:29.744689 19254 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T06:38:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T06:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:29Z Disabled control of system clock
I20260807 06:38:29.782358 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:37815
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:35333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:37815
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/testchainca.cert with env {}
W20260807 06:38:29.865221 19258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:29.865435 19258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:29.865458 19258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:29.866643 19258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:29.866689 19258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:29.866701 19258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:29.866714 19258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:29.866726 19258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:38:29.866739 19258 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:38:29.866755 19258 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 06:38:29.868194 19258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:35333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:37815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:37815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:29.868391 19258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:29.868553 19258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:29.873412 19269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:29.873404 19266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:29.873404 19265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:29.873692 19258 server_base.cc:1034] running on GCE node
I20260807 06:38:29.874001 19258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:29.874334 19258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:29.875497 19258 hybrid_clock.cc:648] HybridClock initialized: now 1786084709875475 us; error 52 us; skew 500 ppm
I20260807 06:38:29.876843 19258 webserver.cc:460] Webserver started at http://127.11.114.254:34795/ using document root <none> and password file <none>
I20260807 06:38:29.877117 19258 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:29.877193 19258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:29.877352 19258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:29.878489 19258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "f0e9ee866ee14f3c80f52e82a5501df0"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "dcebdea1d4750408199e80bf791df2a5"
server_key_iv: "84fd899ded3dbce06eb818d83dd4a469"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.878855 19258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "f0e9ee866ee14f3c80f52e82a5501df0"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "dcebdea1d4750408199e80bf791df2a5"
server_key_iv: "84fd899ded3dbce06eb818d83dd4a469"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.880118 19258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:38:29.880811 19277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:29.880982 19258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:29.881088 19258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "f0e9ee866ee14f3c80f52e82a5501df0"
format_stamp: "Formatted at 2026-08-07 06:38:29 on dist-test-slave-gkw7"
server_key: "dcebdea1d4750408199e80bf791df2a5"
server_key_iv: "84fd899ded3dbce06eb818d83dd4a469"
server_key_version: "encryptionkey@0"
I20260807 06:38:29.881161 19258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:29.902896 19258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:29.903264 19258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:29.903442 19258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:29.907812 19258 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:37815
I20260807 06:38:29.907847 19344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:37815 every 8 connection(s)
I20260807 06:38:29.908231 19258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:29.909231 19345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:29.911557 19345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0: Bootstrap starting.
I20260807 06:38:29.912160 19345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:29.912464 19345 log.cc:824] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:29.913146 19345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0: No bootstrap required, opened a new log
I20260807 06:38:29.914808 19345 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e9ee866ee14f3c80f52e82a5501df0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37815 } }
I20260807 06:38:29.914945 19345 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:29.914968 19345 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e9ee866ee14f3c80f52e82a5501df0, State: Initialized, Role: FOLLOWER
I20260807 06:38:29.915144 19345 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e9ee866ee14f3c80f52e82a5501df0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37815 } }
I20260807 06:38:29.915227 19345 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:29.915279 19345 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:29.915345 19345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:29.915879 19345 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e9ee866ee14f3c80f52e82a5501df0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37815 } }
I20260807 06:38:29.915992 19345 leader_election.cc:302] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0e9ee866ee14f3c80f52e82a5501df0; no voters:
I20260807 06:38:29.916177 19345 leader_election.cc:288] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:29.916255 19351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:29.916431 19351 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [term 1 LEADER]: Becoming Leader. State: Replica: f0e9ee866ee14f3c80f52e82a5501df0, State: Running, Role: LEADER
I20260807 06:38:29.916435 19345 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:29.916585 19351 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e9ee866ee14f3c80f52e82a5501df0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37815 } }
I20260807 06:38:29.917061 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 19258
I20260807 06:38:29.917040 19352 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0e9ee866ee14f3c80f52e82a5501df0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e9ee866ee14f3c80f52e82a5501df0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37815 } } }
I20260807 06:38:29.917129 19352 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:29.917194 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:29.917415 11723 external_mini_cluster.cc:1428] Setting key f6c1f48bfe5f2e2233b4aa955337d88f
I20260807 06:38:29.917816 19360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:29.918273 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0e9ee866ee14f3c80f52e82a5501df0. Latest consensus state: current_term: 1 leader_uuid: "f0e9ee866ee14f3c80f52e82a5501df0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e9ee866ee14f3c80f52e82a5501df0" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37815 } } }
I20260807 06:38:29.918337 19360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:29.918361 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:29.919836 19360 catalog_manager.cc:1347] Generated new cluster ID: 71c7ec16a0004057a1f5972c9585f508
I20260807 06:38:29.919894 19360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:29.927152 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:29.918088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33564 (local address 127.11.114.254:37815)
0807 06:38:29.921678 (+ 3590us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:29.921686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:29.921721 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:38:29.921853 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:29.921860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:29.921927 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:29.922070 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:29.922340 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.923720 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.924900 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.925553 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.925674 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.926422 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:29.926426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:29.926427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:38:29.926641 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:29.926681 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:38:29.926695 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:29.926763 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:29.926859 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 06:38:29.926938 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3509,"thread_start_us":81,"threads_started":1}
I20260807 06:38:29.928109 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:38:29.931864 19360 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:29.932369 19360 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:29.934926 19360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0e9ee866ee14f3c80f52e82a5501df0: Generated new TSK 0
I20260807 06:38:29.935112 19360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:38:30.408962 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:29.941584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33576 (local address 127.11.114.254:37815)
0807 06:38:29.941649 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:29.941651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:29.941751 (+ 100us) server_negotiation.cc:407] Connection header received
0807 06:38:29.941821 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:29.941822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:29.941836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:29.941922 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:38:29.942014 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.942795 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.943571 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.943895 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.943968 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.944122 (+ 154us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:38:30.408732 (+464610us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 06:38:30.408901 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33576: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 06:38:30.409030 19375 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33576: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 06:38:30.409163 19387 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:29.941489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37815 (local address 127.0.0.1:33576)
0807 06:38:29.941684 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:29.941693 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:29.941752 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:29.941920 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:29.941920 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:29.941922 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:38:29.941974 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:29.941976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.942895 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.942896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:29.943488 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:29.943489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:29.943980 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:29.943981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:29.944022 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:29.944027 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:38:30.408874 (+464847us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 06:38:30.409006 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.114.254:37815: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":123,"threads_started":1}
I20260807 06:38:30.409614 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 19258
2026-08-07T06:38:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (659 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (110 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:32 dist-test-slave-gkw7 krb5kdc[19404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:32 dist-test-slave-gkw7 krb5kdc[19404](info): set up 2 sockets
Aug 07 06:38:32 dist-test-slave-gkw7 krb5kdc[19404](info): commencing operation
krb5kdc: starting...
W20260807 06:38:33.535245 11723 mini_kdc.cc:121] Time spent starting KDC: real 2.899s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:33 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:33.664567 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:37033
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:37033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:33.748457 19423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:33.748690 19423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:33.748715 19423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:33.749989 19423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:33.750034 19423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:33.750049 19423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:33.750062 19423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:33.750121 19423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:33.751581 19423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:37033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:37033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:33.751854 19423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:33.752112 19423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:33.757179 19431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:33.757184 19432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:33.757174 19437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:33.757310 19423 server_base.cc:1034] running on GCE node
I20260807 06:38:33.757637 19423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:33.757949 19423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:33.759080 19423 hybrid_clock.cc:648] HybridClock initialized: now 1786084713759066 us; error 31 us; skew 500 ppm
Aug 07 06:38:33 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084713, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:33.764737 19423 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:33.765166 19423 webserver.cc:460] Webserver started at http://127.11.114.254:46117/ using document root <none> and password file <none>
I20260807 06:38:33.765357 19423 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:33.765398 19423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:33.765508 19423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:33.766641 19423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "eb669043bb56424581121d09accb84d2"
format_stamp: "Formatted at 2026-08-07 06:38:33 on dist-test-slave-gkw7"
server_key: "274dfb9c79ca555ce35c04cf93f3b638"
server_key_iv: "2c0ad7f90561867fe75c1d103f0c8002"
server_key_version: "encryptionkey@0"
I20260807 06:38:33.766947 19423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "eb669043bb56424581121d09accb84d2"
format_stamp: "Formatted at 2026-08-07 06:38:33 on dist-test-slave-gkw7"
server_key: "274dfb9c79ca555ce35c04cf93f3b638"
server_key_iv: "2c0ad7f90561867fe75c1d103f0c8002"
server_key_version: "encryptionkey@0"
I20260807 06:38:33.768244 19423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:38:33.768953 19447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:33.769161 19423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:33.769268 19423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "eb669043bb56424581121d09accb84d2"
format_stamp: "Formatted at 2026-08-07 06:38:33 on dist-test-slave-gkw7"
server_key: "274dfb9c79ca555ce35c04cf93f3b638"
server_key_iv: "2c0ad7f90561867fe75c1d103f0c8002"
server_key_version: "encryptionkey@0"
I20260807 06:38:33.769372 19423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:33.792328 19423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:33.794320 19423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:33.794435 19423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:33.798422 19423 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:37033
I20260807 06:38:33.798485 19521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:37033 every 8 connection(s)
I20260807 06:38:33.798858 19423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:33.799789 19522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:33.799922 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 19423
I20260807 06:38:33.800048 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:33.800247 11723 external_mini_cluster.cc:1428] Setting key 0d67d1b653e07f76c9762ee5b9d99c12
I20260807 06:38:33.803285 19522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2: Bootstrap starting.
I20260807 06:38:33.804049 19522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:33.804361 19522 log.cc:824] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:33.805058 19522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2: No bootstrap required, opened a new log
Aug 07 06:38:33 dist-test-slave-gkw7 krb5kdc[19404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:33.807404 19522 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb669043bb56424581121d09accb84d2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37033 } }
I20260807 06:38:33.807586 19522 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:33.807637 19522 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb669043bb56424581121d09accb84d2, State: Initialized, Role: FOLLOWER
I20260807 06:38:33.807778 19522 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb669043bb56424581121d09accb84d2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37033 } }
I20260807 06:38:33.807876 19522 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:33.807914 19522 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:33.807969 19522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:33.808725 19522 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb669043bb56424581121d09accb84d2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37033 } }
I20260807 06:38:33.808866 19522 leader_election.cc:302] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb669043bb56424581121d09accb84d2; no voters:
I20260807 06:38:33.809046 19522 leader_election.cc:288] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:33.809111 19529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:33.809278 19529 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [term 1 LEADER]: Becoming Leader. State: Replica: eb669043bb56424581121d09accb84d2, State: Running, Role: LEADER
I20260807 06:38:33.809326 19522 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:33.809405 19529 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb669043bb56424581121d09accb84d2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37033 } }
I20260807 06:38:33.809758 19529 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb669043bb56424581121d09accb84d2. Latest consensus state: current_term: 1 leader_uuid: "eb669043bb56424581121d09accb84d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb669043bb56424581121d09accb84d2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37033 } } }
I20260807 06:38:33.809854 19529 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:33.809774 19530 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb669043bb56424581121d09accb84d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb669043bb56424581121d09accb84d2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37033 } } }
I20260807 06:38:33.809993 19530 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:33.810072 19538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:33.810586 19538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:33.811721 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:33.800839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.11.114.254:37033)
0807 06:38:33.800998 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:33.801006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:33.801031 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:38:33.801463 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:33.801469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:33.801568 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:33.801701 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:33.802556 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:33.803479 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:33.804553 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:33.804950 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:33.805074 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:33.807863 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:33.807868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:33.807868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:33.807881 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:33.810017 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:33.810492 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:33.810496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:33.810496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:33.810625 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:33.810862 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:33.810863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:33.810863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:33.811152 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:33.811333 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:33.811408 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 06:38:33.811552 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260807 06:38:33.813405 19538 catalog_manager.cc:1347] Generated new cluster ID: 39d503c24e0d4fd8a22ed3826541fbb0
I20260807 06:38:33.813458 19538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:33.822996 19538 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:33.823786 19538 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:33.832607 19538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb669043bb56424581121d09accb84d2: Generated new TSK 0
I20260807 06:38:33.832839 19538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:33.861416 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37033
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:33.946406 19555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:33.946626 19555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:33.946650 19555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:33.947744 19555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:33.947781 19555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:33.947808 19555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:33.949273 19555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:33.949477 19555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:33.949726 19555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:33.955334 19564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:33.955343 19563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:33.955353 19566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:33.955681 19555 server_base.cc:1034] running on GCE node
I20260807 06:38:33.955942 19555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:33.956280 19555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:33.957464 19555 hybrid_clock.cc:648] HybridClock initialized: now 1786084713957451 us; error 31 us; skew 500 ppm
Aug 07 06:38:33 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084713, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:33.960243 19555 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:33.960718 19555 webserver.cc:460] Webserver started at http://127.11.114.193:34245/ using document root <none> and password file <none>
I20260807 06:38:33.960989 19555 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:33.961094 19555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:33.961249 19555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:33.962482 19555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "48610f8e3bcd40089afb78f2138ef00f"
format_stamp: "Formatted at 2026-08-07 06:38:33 on dist-test-slave-gkw7"
server_key: "ac4be6adb1e7afe8c3090a9b8568238e"
server_key_iv: "7bf135d0313eee7cd490a513aaa4b2e2"
server_key_version: "encryptionkey@0"
I20260807 06:38:33.962949 19555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "48610f8e3bcd40089afb78f2138ef00f"
format_stamp: "Formatted at 2026-08-07 06:38:33 on dist-test-slave-gkw7"
server_key: "ac4be6adb1e7afe8c3090a9b8568238e"
server_key_iv: "7bf135d0313eee7cd490a513aaa4b2e2"
server_key_version: "encryptionkey@0"
I20260807 06:38:33.964445 19555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:38:33.965265 19579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:33.965478 19555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:38:33.965597 19555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "48610f8e3bcd40089afb78f2138ef00f"
format_stamp: "Formatted at 2026-08-07 06:38:33 on dist-test-slave-gkw7"
server_key: "ac4be6adb1e7afe8c3090a9b8568238e"
server_key_iv: "7bf135d0313eee7cd490a513aaa4b2e2"
server_key_version: "encryptionkey@0"
I20260807 06:38:33.965679 19555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:33.980468 19555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:33.982797 19555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:33.982954 19555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:33.983222 19555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:33.983564 19555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:33.983621 19555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:33.983704 19555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:33.983737 19555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:33.989622 19555 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:45807
I20260807 06:38:33.989667 19739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:45807 every 8 connection(s)
I20260807 06:38:33.990036 19555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:38:33 dist-test-slave-gkw7 krb5kdc[19404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084713, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:33.996799 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 19555
I20260807 06:38:33.996929 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:33.997159 11723 external_mini_cluster.cc:1428] Setting key 8661cc879bcd85c2e92320b1af4209a4
I20260807 06:38:34.000025 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:33.990848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:47499 (local address 127.11.114.254:37033)
0807 06:38:33.990944 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:33.990946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:33.991189 (+ 243us) server_negotiation.cc:407] Connection header received
0807 06:38:33.992114 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:33.992116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:33.992128 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:33.992193 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:33.992859 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:33.993373 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:33.994368 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:33.994570 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:33.994626 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:33.996822 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:33.996824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:33.996825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:33.996834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:33.998475 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:33.998921 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:33.998922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:33.998923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:33.998995 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:33.999275 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:33.999276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:33.999277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:33.999445 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:33.999514 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:33.999796 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 06:38:33.999897 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:38:34.000047 19743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:33.990785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37033 (local address 127.11.114.193:47499)
0807 06:38:33.991031 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:33.991075 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:33.992008 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:33.992282 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:33.992286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:33.992552 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:33.992761 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:33.992765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:33.993457 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:33.993458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:33.994296 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:33.994298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:33.994645 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:33.994646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:33.994765 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:33.995241 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:33.995249 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:33.996724 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:33.998583 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:33.998591 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:33.998593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:33.998838 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:33.999069 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:33.999070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:33.999071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:33.999196 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:33.999522 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:33.999529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:33.999658 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 06:38:33.999751 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 06:38:33.999880 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":99,"threads_started":1}
I20260807 06:38:34.000666 19741 heartbeater.cc:347] Connected to a master server at 127.11.114.254:37033
I20260807 06:38:34.000802 19741 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:34.000998 19741 heartbeater.cc:511] Master 127.11.114.254:37033 requested a full tablet report, sending...
I20260807 06:38:34.001628 19472 ts_manager.cc:194] Registered new tserver with Master: 48610f8e3bcd40089afb78f2138ef00f (127.11.114.193:45807)
I20260807 06:38:34.002331 19472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:47499
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:34.026041 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37033
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:34.109758 19746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:34.109994 19746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:34.110034 19746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:34.111251 19746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:34.111318 19746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:34.111363 19746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:34.112872 19746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:34.113157 19746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:34.113394 19746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:34.118444 19758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:34.118438 19761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:34.118422 19755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:34.118811 19746 server_base.cc:1034] running on GCE node
I20260807 06:38:34.119033 19746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:34.119272 19746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:34.120428 19746 hybrid_clock.cc:648] HybridClock initialized: now 1786084714120419 us; error 31 us; skew 500 ppm
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.122936 19746 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:34.123314 19746 webserver.cc:460] Webserver started at http://127.11.114.194:34641/ using document root <none> and password file <none>
I20260807 06:38:34.123509 19746 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:34.123549 19746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:34.123636 19746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:34.124611 19746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "833db96fa46c4ab3979820d3074d379b"
format_stamp: "Formatted at 2026-08-07 06:38:34 on dist-test-slave-gkw7"
server_key: "097f60c43937e9728708030fd588dc8a"
server_key_iv: "7d5f6424567e50a7f3b39ad5a8400c8f"
server_key_version: "encryptionkey@0"
I20260807 06:38:34.124907 19746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "833db96fa46c4ab3979820d3074d379b"
format_stamp: "Formatted at 2026-08-07 06:38:34 on dist-test-slave-gkw7"
server_key: "097f60c43937e9728708030fd588dc8a"
server_key_iv: "7d5f6424567e50a7f3b39ad5a8400c8f"
server_key_version: "encryptionkey@0"
I20260807 06:38:34.126276 19746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:34.126967 19772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:34.127174 19746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:34.127262 19746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "833db96fa46c4ab3979820d3074d379b"
format_stamp: "Formatted at 2026-08-07 06:38:34 on dist-test-slave-gkw7"
server_key: "097f60c43937e9728708030fd588dc8a"
server_key_iv: "7d5f6424567e50a7f3b39ad5a8400c8f"
server_key_version: "encryptionkey@0"
I20260807 06:38:34.127316 19746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:34.139978 19746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:34.141891 19746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:34.142002 19746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:34.142184 19746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:34.142445 19746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:34.142478 19746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:34.142500 19746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:34.142513 19746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:34.148226 19746 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:32951
I20260807 06:38:34.148310 19935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:32951 every 8 connection(s)
I20260807 06:38:34.148628 19746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:34.151196 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 19746
I20260807 06:38:34.151330 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:34.151520 11723 external_mini_cluster.cc:1428] Setting key 23554aee131dc358ad222925ffa2f6a0
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:34.158937 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:34.149437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:58661 (local address 127.11.114.254:37033)
0807 06:38:34.149599 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:34.149602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:34.149782 (+ 180us) server_negotiation.cc:407] Connection header received
0807 06:38:34.150665 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:34.150666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:34.150678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:34.150742 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:34.151579 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.152125 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.153183 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.153405 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.153494 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:34.155820 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:34.155822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:34.155822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:34.155830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:34.157499 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.157934 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.157935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.157936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.158004 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.158263 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.158265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.158265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.158410 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:34.158487 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:34.158736 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 06:38:34.158838 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 06:38:34.158970 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:34.149340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37033 (local address 127.11.114.194:58661)
0807 06:38:34.149643 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:34.149690 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:34.150569 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:34.150828 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:34.150831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:34.151163 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:34.151449 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:34.151456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.152235 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.152237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:34.153074 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:34.153077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.153506 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.153507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:34.153662 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:34.154289 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:34.154298 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:34.155726 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:34.157606 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:34.157612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:34.157614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:34.157854 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:34.158084 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:34.158085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:34.158085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:34.158186 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:34.158481 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:34.158486 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:34.158603 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 06:38:34.158687 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 06:38:34.158807 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260807 06:38:34.159485 19936 heartbeater.cc:347] Connected to a master server at 127.11.114.254:37033
I20260807 06:38:34.159588 19936 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:34.159780 19936 heartbeater.cc:511] Master 127.11.114.254:37033 requested a full tablet report, sending...
I20260807 06:38:34.160264 19472 ts_manager.cc:194] Registered new tserver with Master: 833db96fa46c4ab3979820d3074d379b (127.11.114.194:32951)
I20260807 06:38:34.160791 19472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:58661
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:34.180007 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37033
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:34.264394 19943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:34.264650 19943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:34.264693 19943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:34.265944 19943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:34.266016 19943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:34.266059 19943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:34.267580 19943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:34.267863 19943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:34.268102 19943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:34.273593 19953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:34.273593 19949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:34.273650 19950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:34.274034 19943 server_base.cc:1034] running on GCE node
I20260807 06:38:34.274261 19943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:34.274505 19943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:34.275666 19943 hybrid_clock.cc:648] HybridClock initialized: now 1786084714275665 us; error 48 us; skew 500 ppm
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.278242 19943 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:34.278759 19943 webserver.cc:460] Webserver started at http://127.11.114.195:34341/ using document root <none> and password file <none>
I20260807 06:38:34.279011 19943 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:34.279081 19943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:34.279239 19943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:34.280320 19943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "6ebcd42d402e4bf29e73d2fc7598f9ee"
format_stamp: "Formatted at 2026-08-07 06:38:34 on dist-test-slave-gkw7"
server_key: "851e553290d21083f59454c557408d47"
server_key_iv: "2ee493838c9220456cb7caa250a43693"
server_key_version: "encryptionkey@0"
I20260807 06:38:34.280674 19943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "6ebcd42d402e4bf29e73d2fc7598f9ee"
format_stamp: "Formatted at 2026-08-07 06:38:34 on dist-test-slave-gkw7"
server_key: "851e553290d21083f59454c557408d47"
server_key_iv: "2ee493838c9220456cb7caa250a43693"
server_key_version: "encryptionkey@0"
I20260807 06:38:34.282199 19943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:34.282943 19962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:34.283131 19943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:34.283247 19943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "6ebcd42d402e4bf29e73d2fc7598f9ee"
format_stamp: "Formatted at 2026-08-07 06:38:34 on dist-test-slave-gkw7"
server_key: "851e553290d21083f59454c557408d47"
server_key_iv: "2ee493838c9220456cb7caa250a43693"
server_key_version: "encryptionkey@0"
I20260807 06:38:34.283322 19943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:34.297257 19943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:34.299492 19943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:34.299666 19943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:34.299908 19943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:34.300226 19943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:34.300287 19943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:34.300321 19943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:34.300348 19943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:34.306504 19943 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:43115
I20260807 06:38:34.306559 20126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:43115 every 8 connection(s)
I20260807 06:38:34.306871 19943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:34.315467 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 19943
I20260807 06:38:34.315567 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:34.315757 11723 external_mini_cluster.cc:1428] Setting key af347f18baf83aa9dfbe7eef7d6aa76d
I20260807 06:38:34.317163 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:34.307616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:45595 (local address 127.11.114.254:37033)
0807 06:38:34.307693 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:34.307695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:34.307878 (+ 183us) server_negotiation.cc:407] Connection header received
0807 06:38:34.308948 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:34.308950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:34.308963 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:34.309031 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:34.309742 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.310344 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.311446 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.311714 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.311793 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:34.314009 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:34.314011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:34.314011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:34.314018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:34.315623 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.316137 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.316139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.316139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.316210 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.316519 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.316520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.316520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.316658 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:34.316748 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:34.317032 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 06:38:34.317098 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 06:38:34.317314 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:34.307582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37033 (local address 127.11.114.195:45595)
0807 06:38:34.307759 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:34.307798 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:34.308829 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:34.309110 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:34.309114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:34.309374 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:34.309638 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:34.309643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.310441 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.310442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:34.311351 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:34.311353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.311810 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.311811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:34.311926 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:34.312357 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:34.312366 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:34.313920 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:34.315718 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:34.315724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:34.315726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:34.316067 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:34.316284 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:34.316285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:34.316285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:34.316450 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:34.316750 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:34.316757 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:34.316909 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 06:38:34.316996 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:38:34.317116 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260807 06:38:34.317812 20128 heartbeater.cc:347] Connected to a master server at 127.11.114.254:37033
I20260807 06:38:34.317921 20128 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:34.318097 20128 heartbeater.cc:511] Master 127.11.114.254:37033 requested a full tablet report, sending...
I20260807 06:38:34.318459 19472 ts_manager.cc:194] Registered new tserver with Master: 6ebcd42d402e4bf29e73d2fc7598f9ee (127.11.114.195:43115)
I20260807 06:38:34.318833 19472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:45595
I20260807 06:38:34.327651 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/test-admin.keytab.
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.344162 11723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 06:38:34.345887 20139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:34.345047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.11.114.193:45807)
0807 06:38:34.345236 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:34.345247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:34.345258 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:38:34.345285 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:34.345287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:34.345300 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:34.345372 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:34.345796 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58856: BlockingRecv error: recv got EOF from 127.0.0.1:58856 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
W20260807 06:38:34.346051 11723 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc)
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.346989 11723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.347844 11723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:34.356119 20139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:34.348367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58860 (local address 127.11.114.193:45807)
0807 06:38:34.348422 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:34.348424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:34.348434 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:34.348519 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:34.348520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:34.348531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:34.348601 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:34.349000 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.349803 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.350542 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.350955 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.351070 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:34.353723 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:34.353732 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:34.353732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:34.353739 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:34.355138 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.355456 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.355459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.355460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.355514 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.355695 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.355696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.355696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.355813 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:34.355905 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:34.355991 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 06:38:34.356050 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 06:38:34.357995 20139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:34.357349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58868 (local address 127.11.114.193:45807)
0807 06:38:34.357387 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:34.357389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:34.357431 (+ 42us) server_negotiation.cc:407] Connection header received
0807 06:38:34.357477 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:34.357478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:34.357485 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:34.357573 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:34.357945 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58868: BlockingRecv error: recv got EOF from 127.0.0.1:58868 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 06:38:34.358068 11723 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc' of type FILE
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.359001 11723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.359784 11723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:34.366526 20139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:34.360250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.11.114.193:45807)
0807 06:38:34.360305 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:34.360306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:34.360316 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:34.360367 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:34.360368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:34.360374 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:34.360417 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:34.360721 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.361200 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.361900 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.362089 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.362142 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:34.363957 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:34.363958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:34.363959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:34.363966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:34.365322 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.365770 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.365771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.365772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.365850 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.366052 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.366052 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.366053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.366150 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:34.366246 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:34.366355 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 06:38:34.366431 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 06:38:34.367865 20139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:34.367461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.11.114.193:45807)
0807 06:38:34.367496 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:34.367498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:34.367547 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:38:34.367589 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:34.367590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:34.367597 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:34.367639 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:34.367836 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58888: BlockingRecv error: recv got EOF from 127.0.0.1:58888 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 06:38:34.367961 11723 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084684348861-11723-0/krb5kdc/krb5cc)
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.368831 11723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:34.369700 11723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[19404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:38:34.375885 20139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:34.370134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.11.114.193:45807)
0807 06:38:34.370173 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:34.370176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:34.370183 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:38:34.370223 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:34.370224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:34.370242 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:34.370285 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:34.370596 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.371016 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.371738 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:34.371905 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:34.371954 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:34.373879 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:34.373880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:34.373880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:34.373886 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:34.374963 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.375263 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.375264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.375264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.375309 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:34.375465 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:34.375465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:34.375465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:34.375542 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:34.375604 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:34.375699 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 06:38:34.375752 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 06:38:34.376479 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 19555
I20260807 06:38:34.390829 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 19746
I20260807 06:38:34.404124 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 19943
I20260807 06:38:34.414140 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 19423
2026-08-07T06:38:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3800 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[20156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[20156](info): set up 2 sockets
Aug 07 06:38:34 dist-test-slave-gkw7 krb5kdc[20156](info): commencing operation
krb5kdc: starting...
W20260807 06:38:35.635090 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:35 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.254.keytab.
Entry for principal oryx/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.254.keytab.
I20260807 06:38:35.760428 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33755
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.254.keytab
--principal=oryx/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:33755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:35.841958 20175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:35.842196 20175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:35.842243 20175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:35.843369 20175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:35.843434 20175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:35.843461 20175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:35.843492 20175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:35.843516 20175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:35.844985 20175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:33755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.254.keytab
--principal=oryx/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:35.845266 20175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:35.845487 20175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:35.850379 20183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:35.850368 20181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:35.850369 20180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:35.850483 20175 server_base.cc:1034] running on GCE node
I20260807 06:38:35.850889 20175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:35.851204 20175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:35.852392 20175 hybrid_clock.cc:648] HybridClock initialized: now 1786084715852360 us; error 42 us; skew 500 ppm
Aug 07 06:38:35 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084715, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:35.855263 20175 init.cc:377] Logged in from keytab as oryx/127.11.114.254@KRBTEST.COM (short username oryx)
I20260807 06:38:35.855746 20175 webserver.cc:460] Webserver started at http://127.11.114.254:36925/ using document root <none> and password file <none>
I20260807 06:38:35.856014 20175 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:35.856071 20175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:35.856249 20175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:35.857383 20175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d"
format_stamp: "Formatted at 2026-08-07 06:38:35 on dist-test-slave-gkw7"
server_key: "048024f07f2fa33cdda3bf2b3b840791"
server_key_iv: "aa62896c81818dc54aa10057d7a268c1"
server_key_version: "encryptionkey@0"
I20260807 06:38:35.857831 20175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d"
format_stamp: "Formatted at 2026-08-07 06:38:35 on dist-test-slave-gkw7"
server_key: "048024f07f2fa33cdda3bf2b3b840791"
server_key_iv: "aa62896c81818dc54aa10057d7a268c1"
server_key_version: "encryptionkey@0"
I20260807 06:38:35.859308 20175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:35.860047 20193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:35.860225 20175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:35.860347 20175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d"
format_stamp: "Formatted at 2026-08-07 06:38:35 on dist-test-slave-gkw7"
server_key: "048024f07f2fa33cdda3bf2b3b840791"
server_key_iv: "aa62896c81818dc54aa10057d7a268c1"
server_key_version: "encryptionkey@0"
I20260807 06:38:35.860421 20175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:35.874200 20175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:35.876534 20175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:35.876713 20175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:35.880957 20175 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:33755
I20260807 06:38:35.880990 20263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:33755 every 8 connection(s)
I20260807 06:38:35.881371 20175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:35.882460 20264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:35.884723 20264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: Bootstrap starting.
I20260807 06:38:35.885327 20264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:35.885509 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 20175
I20260807 06:38:35.885648 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:35.885707 20264 log.cc:824] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:35.885902 11723 external_mini_cluster.cc:1428] Setting key 2eaa0eda55058916f789950111ae2dbb
I20260807 06:38:35.886642 20264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: No bootstrap required, opened a new log
I20260807 06:38:35.889266 20264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } }
I20260807 06:38:35.889439 20264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:35.889477 20264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27284d9f9c8e4ff4a0c93a466cc3af4d, State: Initialized, Role: FOLLOWER
I20260807 06:38:35.889634 20264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } }
I20260807 06:38:35.889729 20264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:35.889761 20264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:35.889806 20264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:35.890463 20264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } }
I20260807 06:38:35.890558 20264 leader_election.cc:302] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27284d9f9c8e4ff4a0c93a466cc3af4d; no voters:
I20260807 06:38:35.890730 20264 leader_election.cc:288] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:35.890764 20271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:35.890882 20271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 1 LEADER]: Becoming Leader. State: Replica: 27284d9f9c8e4ff4a0c93a466cc3af4d, State: Running, Role: LEADER
I20260807 06:38:35.891009 20264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:35.891021 20271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } }
I20260807 06:38:35.892050 20275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27284d9f9c8e4ff4a0c93a466cc3af4d. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } }
I20260807 06:38:35.892138 20275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
Aug 07 06:38:35 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:35.892563 20274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } }
I20260807 06:38:35.893640 20274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
W20260807 06:38:35.893095 20301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:38:35.893705 20301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:35.893790 20295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:35.893960 20295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:35.896028 20295 catalog_manager.cc:1347] Generated new cluster ID: 87ae0a6e997546f9bdd84ff1a1d87d4c
I20260807 06:38:35.896098 20295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:35.898314 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:35.886510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.11.114.254:33755)
0807 06:38:35.886717 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:35.886727 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:35.886762 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:38:35.887203 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:35.887212 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:35.887268 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:35.887386 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:35.888177 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:35.889045 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:35.890228 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:35.890760 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:35.890945 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:35.894097 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:35.894101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:35.894102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:35.894115 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:35.896553 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:35.896971 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:35.896974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:35.896975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:35.897080 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:35.897317 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:35.897318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:35.897319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:35.897651 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:35.897838 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:35.897926 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 06:38:35.898067 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260807 06:38:35.910668 20295 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:35.911401 20295 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:35.925947 20295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: Generated new TSK 0
I20260807 06:38:35.926172 20295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab.
Entry for principal oryx/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab.
I20260807 06:38:35.960464 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33755
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab
--principal=oryx/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:36.045768 20310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:36.045966 20310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:36.045990 20310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:36.047222 20310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:36.047267 20310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:36.047294 20310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:36.048728 20310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab
--principal=oryx/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:36.048910 20310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:36.049109 20310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:36.054258 20317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.054265 20320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.054268 20318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.054484 20310 server_base.cc:1034] running on GCE node
I20260807 06:38:36.054754 20310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:36.054963 20310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:36.056100 20310 hybrid_clock.cc:648] HybridClock initialized: now 1786084716056080 us; error 38 us; skew 500 ppm
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:36.058677 20310 init.cc:377] Logged in from keytab as oryx/127.11.114.193@KRBTEST.COM (short username oryx)
I20260807 06:38:36.059106 20310 webserver.cc:460] Webserver started at http://127.11.114.193:35893/ using document root <none> and password file <none>
I20260807 06:38:36.059358 20310 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.059403 20310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.059571 20310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:36.060617 20310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "110dec602478421ba7f0fd9d6806ab03"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "f14911dfb54c62c1d01545ab1f48eb80"
server_key_iv: "8801b3c84f9746d5326b5803ff44fae9"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.060978 20310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "110dec602478421ba7f0fd9d6806ab03"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "f14911dfb54c62c1d01545ab1f48eb80"
server_key_iv: "8801b3c84f9746d5326b5803ff44fae9"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.062469 20310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:36.063282 20328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.063463 20310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:36.063549 20310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "110dec602478421ba7f0fd9d6806ab03"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "f14911dfb54c62c1d01545ab1f48eb80"
server_key_iv: "8801b3c84f9746d5326b5803ff44fae9"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.063602 20310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.074380 20310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:36.076345 20310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:36.076455 20310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:36.076680 20310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:36.076998 20310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:36.077055 20310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.077097 20310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:36.077131 20310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.083220 20310 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:44647
I20260807 06:38:36.083325 20504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:44647 every 8 connection(s)
I20260807 06:38:36.083635 20310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:36.086018 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 20310
I20260807 06:38:36.086102 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:36.086313 11723 external_mini_cluster.cc:1428] Setting key db633bf59f6648ebfa3f6f813562c1aa
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:36.094980 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.084444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:48559 (local address 127.11.114.254:33755)
0807 06:38:36.084494 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.084496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.084731 (+ 235us) server_negotiation.cc:407] Connection header received
0807 06:38:36.085214 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.085215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.085236 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.085285 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.086498 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.087201 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.088334 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.088590 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.088677 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.091569 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.091570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.091570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.091578 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.093241 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.093864 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.093864 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.093865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.093921 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.094276 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.094276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.094277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.094382 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.094431 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.094825 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.094911 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:38:36.095309 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.084377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:48559)
0807 06:38:36.084582 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.084620 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.085127 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.085349 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.085352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.086038 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.086318 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.086324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.087306 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.087307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.088228 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.088231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.088699 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.088700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.088845 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.089453 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.089464 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.091470 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.093339 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.093347 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.093350 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.093774 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.093993 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.093995 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.093995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.094196 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.094454 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.094463 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.094667 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.094796 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.094996 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260807 06:38:36.095922 20506 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33755
I20260807 06:38:36.096051 20506 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:36.096298 20506 heartbeater.cc:511] Master 127.11.114.254:33755 requested a full tablet report, sending...
I20260807 06:38:36.096853 20217 ts_manager.cc:194] Registered new tserver with Master: 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647)
WARNING: no policy specified for oryx/127.11.114.194@KRBTEST.COM; defaulting to no policy
I20260807 06:38:36.097481 20217 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:48559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab.
Entry for principal oryx/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab.
I20260807 06:38:36.116484 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33755
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab
--principal=oryx/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:36.199291 20512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:36.199509 20512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:36.199532 20512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:36.200600 20512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:36.200635 20512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:36.200660 20512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:36.202106 20512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab
--principal=oryx/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:36.202302 20512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:36.202481 20512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:36.207593 20519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.207557 20521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.207763 20512 server_base.cc:1034] running on GCE node
W20260807 06:38:36.207795 20518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.208029 20512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:36.208330 20512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:36.209502 20512 hybrid_clock.cc:648] HybridClock initialized: now 1786084716209487 us; error 46 us; skew 500 ppm
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:36.212311 20512 init.cc:377] Logged in from keytab as oryx/127.11.114.194@KRBTEST.COM (short username oryx)
I20260807 06:38:36.212790 20512 webserver.cc:460] Webserver started at http://127.11.114.194:41923/ using document root <none> and password file <none>
I20260807 06:38:36.213047 20512 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.213125 20512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.213315 20512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:36.214473 20512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "31ddfc1398ad41d98c620c704c2e2ad4"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "41f6d445c42515691c13c1675c652b80"
server_key_iv: "62cf7f5d3c254bbf9743c120bf95a41b"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.214864 20512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "31ddfc1398ad41d98c620c704c2e2ad4"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "41f6d445c42515691c13c1675c652b80"
server_key_iv: "62cf7f5d3c254bbf9743c120bf95a41b"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.216261 20512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:36.217051 20533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.217221 20512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:36.217329 20512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "31ddfc1398ad41d98c620c704c2e2ad4"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "41f6d445c42515691c13c1675c652b80"
server_key_iv: "62cf7f5d3c254bbf9743c120bf95a41b"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.217404 20512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.224994 20512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:36.227308 20512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:36.227479 20512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:36.227708 20512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:36.228024 20512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:36.228083 20512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.228122 20512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:36.228150 20512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.234139 20512 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:37155
I20260807 06:38:36.234175 20696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:37155 every 8 connection(s)
I20260807 06:38:36.234534 20512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:36.241616 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 20512
I20260807 06:38:36.241731 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:36.241958 11723 external_mini_cluster.cc:1428] Setting key 6bdcfe6fee0f3f433639eb4d764f01aa
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:36.245790 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.235291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:44917 (local address 127.11.114.254:33755)
0807 06:38:36.235389 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.235392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.235671 (+ 279us) server_negotiation.cc:407] Connection header received
0807 06:38:36.236174 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.236175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.236187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.236256 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.237186 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.237837 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.239418 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.239670 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.239752 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.242714 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.242716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.242716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.242725 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.244244 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.244701 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.244702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.244703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.244770 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.245042 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.245043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.245044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.245174 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.245263 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.245603 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.245697 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 06:38:36.246006 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.235246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:44917)
0807 06:38:36.235524 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.235564 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.236059 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.236312 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.236315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.236893 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.237090 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.237095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.238000 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.238002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.239321 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.239324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.239829 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.239831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.239986 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.240642 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.240651 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.242632 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.244353 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.244360 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.244362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.244599 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.244841 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.244842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.244842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.244963 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.245267 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.245275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.245418 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.245529 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.245745 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":153,"threads_started":1}
I20260807 06:38:36.246541 20701 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33755
I20260807 06:38:36.246672 20701 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:36.246861 20701 heartbeater.cc:511] Master 127.11.114.254:33755 requested a full tablet report, sending...
I20260807 06:38:36.247308 20217 ts_manager.cc:194] Registered new tserver with Master: 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155)
I20260807 06:38:36.247704 20217 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:44917
WARNING: no policy specified for oryx/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab.
Entry for principal oryx/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab.
I20260807 06:38:36.269239 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33755
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab
--principal=oryx/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:36.352093 20712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:36.352300 20712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:36.352324 20712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:36.353387 20712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:36.353422 20712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:36.353446 20712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:36.354967 20712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab
--principal=oryx/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:33755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:36.355154 20712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:36.355333 20712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:36.360267 20727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.360311 20722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.360278 20721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.360386 20712 server_base.cc:1034] running on GCE node
I20260807 06:38:36.360734 20712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:36.360992 20712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:36.362156 20712 hybrid_clock.cc:648] HybridClock initialized: now 1786084716362149 us; error 39 us; skew 500 ppm
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:36.364923 20712 init.cc:377] Logged in from keytab as oryx/127.11.114.195@KRBTEST.COM (short username oryx)
I20260807 06:38:36.365378 20712 webserver.cc:460] Webserver started at http://127.11.114.195:38199/ using document root <none> and password file <none>
I20260807 06:38:36.365684 20712 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.365756 20712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.365917 20712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:36.366998 20712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "0ecbfc604f3d4bc29742140f57361c18"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "02e3197be2adcc17754b7aea3880d655"
server_key_iv: "e611dac009edb6e6e5b2986eec4462cd"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.367368 20712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "0ecbfc604f3d4bc29742140f57361c18"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "02e3197be2adcc17754b7aea3880d655"
server_key_iv: "e611dac009edb6e6e5b2986eec4462cd"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.368690 20712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:36.369449 20742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.369639 20712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:36.369740 20712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "0ecbfc604f3d4bc29742140f57361c18"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "02e3197be2adcc17754b7aea3880d655"
server_key_iv: "e611dac009edb6e6e5b2986eec4462cd"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.369822 20712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.376533 20712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:36.378654 20712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:36.378808 20712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:36.379009 20712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:36.379307 20712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:36.379339 20712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.379359 20712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:36.379405 20712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.385210 20712 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:38629
I20260807 06:38:36.385308 20895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:38629 every 8 connection(s)
I20260807 06:38:36.385686 20712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:36.394668 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 20712
I20260807 06:38:36.394778 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:36.394773 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.386376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:39893 (local address 127.11.114.254:33755)
0807 06:38:36.386419 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.386421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.386645 (+ 224us) server_negotiation.cc:407] Connection header received
0807 06:38:36.387154 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.387155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.387168 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.387221 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.388150 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.388671 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.389731 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.389952 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.390030 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.392148 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.392149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.392150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.392158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.393573 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.393938 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.393938 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.393939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.393989 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.394208 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.394208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.394209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.394314 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.394363 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.394585 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.394667 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:38:36.394917 20898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.386364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:39893)
0807 06:38:36.386531 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.386564 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.387053 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.387287 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.387291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.387865 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.388042 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.388046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.388766 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.388767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.389633 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.389636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.390035 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.390037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.390178 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.390573 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.390580 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.392058 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.393651 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.393656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.393658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.393868 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.394046 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.394047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.394047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.394139 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.394368 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.394372 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.394478 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.394556 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.394667 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260807 06:38:36.395010 11723 external_mini_cluster.cc:1428] Setting key 28c93351c887e63d5f6150c012aafc7f
I20260807 06:38:36.395478 20896 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33755
I20260807 06:38:36.395599 20896 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:36.395784 20896 heartbeater.cc:511] Master 127.11.114.254:33755 requested a full tablet report, sending...
I20260807 06:38:36.396173 20217 ts_manager.cc:194] Registered new tserver with Master: 0ecbfc604f3d4bc29742140f57361c18 (127.11.114.195:38629)
I20260807 06:38:36.396497 20217 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:39893
I20260807 06:38:36.406734 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.114.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.253.keytab.
Entry for principal oryx/127.11.114.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.253.keytab.
I20260807 06:38:36.433029 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.253:39293
--webserver_interface=127.11.114.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.253.keytab
--principal=oryx/127.11.114.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:33755,127.11.114.253:39293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:36.516252 20902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:36.516466 20902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:36.516489 20902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:36.517669 20902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:36.517716 20902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:36.517731 20902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:36.517745 20902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:36.517756 20902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:36.519165 20902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:33755,127.11.114.253:39293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.253.keytab
--principal=oryx/127.11.114.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.253:39293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.114.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
W20260807 06:38:36.519263 20902 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.114.254:33755,127.11.114.253:39293'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 06:38:36.519421 20902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:36.519603 20902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:36.524559 20913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.524605 20915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.524683 20919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.524837 20902 server_base.cc:1034] running on GCE node
I20260807 06:38:36.525085 20902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:36.525413 20902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:36.526574 20902 hybrid_clock.cc:648] HybridClock initialized: now 1786084716526572 us; error 40 us; skew 500 ppm
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:36.529089 20902 init.cc:377] Logged in from keytab as oryx/127.11.114.253@KRBTEST.COM (short username oryx)
I20260807 06:38:36.529634 20902 webserver.cc:460] Webserver started at http://127.11.114.253:40787/ using document root <none> and password file <none>
I20260807 06:38:36.529824 20902 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.529865 20902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.529955 20902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:36.530925 20902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data/instance:
uuid: "bbd1d4d07d1e4bd885041290cffa1e1b"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "533ef7f1def0d4e0e9cb744692a50d90"
server_key_iv: "4d25ec96e865d29d99f38e0279d52951"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.531226 20902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal/instance:
uuid: "bbd1d4d07d1e4bd885041290cffa1e1b"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "533ef7f1def0d4e0e9cb744692a50d90"
server_key_iv: "4d25ec96e865d29d99f38e0279d52951"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.532593 20902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:36.533326 20933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.533519 20902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:36.533650 20902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
uuid: "bbd1d4d07d1e4bd885041290cffa1e1b"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "533ef7f1def0d4e0e9cb744692a50d90"
server_key_iv: "4d25ec96e865d29d99f38e0279d52951"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.533706 20902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.552302 20902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:36.554453 20902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:36.554610 20902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:36.559226 20902 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.253:39293
I20260807 06:38:36.559307 21001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.253:39293 every 8 connection(s)
I20260807 06:38:36.559705 20902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
I20260807 06:38:36.560758 21002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:36.562419 21002 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.253@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:36.568243 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 20902
I20260807 06:38:36.571794 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.563202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48838 (local address 127.11.114.254:33755)
0807 06:38:36.563259 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.563262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.563450 (+ 188us) server_negotiation.cc:407] Connection header received
0807 06:38:36.563961 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.563962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.563976 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.564059 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.564664 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.565155 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.566317 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.566499 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.566554 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.568860 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.568861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.568862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.568869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.570327 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.570753 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.570753 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.570754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.570803 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.571048 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.571049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.571049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.571278 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.571362 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.571591 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.571673 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:38:36.571862 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.563114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:48838)
0807 06:38:36.563326 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.563363 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.563849 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.564147 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.564150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.564396 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.564588 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.564591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.565231 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.565232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.566243 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.566246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.566569 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.566570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.566666 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.567230 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.567238 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.568703 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.570403 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.570408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.570410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.570682 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.570866 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.570867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.570867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.570988 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.571360 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.571364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.571469 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.571546 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.571655 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
I20260807 06:38:36.572592 21002 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.114.253@KRBTEST.COM
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.253@KRBTEST.COM for oryx/127.11.114.253@KRBTEST.COM
I20260807 06:38:36.578445 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.568751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55366 (local address 127.11.114.253:39293)
0807 06:38:36.568921 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.568926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.568974 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:38:36.569033 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.569035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.569049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.569131 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.569958 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.570812 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.572250 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.572722 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.572796 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.575486 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.575493 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.575494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.575501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.577180 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.577634 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.577637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.577637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.577711 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.577941 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.577941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.577942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.578063 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.578172 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.578297 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.578377 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260807 06:38:36.580341 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.572828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.0.0.1:55374)
0807 06:38:36.572869 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.572883 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.572909 (+ 26us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.573174 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.573175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.573338 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.573399 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.573402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.574098 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.574099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.574695 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.574698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.575169 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.575170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.575252 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.575633 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.575637 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.576921 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.579368 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.579369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.579369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.579530 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.579743 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.579744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.579744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.579798 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.580036 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.580036 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.580072 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.580139 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.580236 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 06:38:36.580423 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.572881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55374 (local address 127.11.114.253:39293)
0807 06:38:36.573048 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.573052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.573060 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:36.573078 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.573079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.573089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.573157 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.573482 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.574022 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.574776 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.575097 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.575149 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.577005 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.577006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.577006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.577013 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.579219 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.579612 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.579613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.579613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.579669 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.579867 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.579868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.579868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.579969 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.580043 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.580165 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.580284 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260807 06:38:36.581681 21002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Bootstrap starting.
I20260807 06:38:36.582388 21002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:36.582672 21002 log.cc:824] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:36.583386 21002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: No bootstrap required, opened a new log
I20260807 06:38:36.584846 21002 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } }
I20260807 06:38:36.584962 21002 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:36.585007 21002 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd1d4d07d1e4bd885041290cffa1e1b, State: Initialized, Role: FOLLOWER
I20260807 06:38:36.585148 21002 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } }
I20260807 06:38:36.585407 21014 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } } }
I20260807 06:38:36.585453 21002 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:36.585500 21014 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:38:36.587365 21030 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:38:36.587426 21030 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:36.589257 20902 master_runner.cc:428] Detected that this master bbd1d4d07d1e4bd885041290cffa1e1b is joining an existing cluster
I20260807 06:38:36.589318 20902 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.114.253:39293
I20260807 06:38:36.596832 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.590385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48844 (local address 127.11.114.254:33755)
0807 06:38:36.590432 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.590434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.590635 (+ 201us) server_negotiation.cc:407] Connection header received
0807 06:38:36.590723 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.590724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.590742 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.590790 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.591177 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.591753 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.592556 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.592766 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.592848 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.593854 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.593856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.593856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.593863 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.595442 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.596017 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.596019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.596019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.596076 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.596314 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.596315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.596315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.596447 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.596508 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.596667 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.596747 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:38:36.596841 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.590343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:48844)
0807 06:38:36.590536 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.590547 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.590657 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.590793 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.590794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.590996 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.591069 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.591071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.591835 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.591836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.592475 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.592477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.592850 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.592851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.592917 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.593360 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.593364 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.593772 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.595651 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.595652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.595653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.595943 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.596157 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.596158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.596158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.596245 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.596499 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.596500 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.596551 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.596643 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.596739 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"spinlock_wait_cycles":8576,"thread_start_us":104,"threads_started":1}
I20260807 06:38:36.597360 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.590540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.11.114.253:39293)
0807 06:38:36.590572 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.590573 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.590581 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:36.590642 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.590643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.590653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.590719 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.591163 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.591905 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.592928 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.593226 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.593265 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.594255 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.594256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.594257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.594262 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.596037 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.596434 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.596434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.596435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.596490 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.596668 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.596668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.596669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.596784 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.596823 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.597205 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.597280 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:38:36.597841 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.590385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.0.0.1:55384)
0807 06:38:36.590519 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.590530 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.590584 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.590757 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.590758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.590999 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.591078 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.591080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.591991 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.591992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.592838 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.592841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.593332 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.593333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.593389 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.593806 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.593811 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.594164 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.596140 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.596140 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.596141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.596361 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.596561 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.596561 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.596562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.596608 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.596864 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.596864 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.597101 (+ 237us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.597599 (+ 498us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.597732 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084715, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.254@KRBTEST.COM for oryx/127.11.114.253@KRBTEST.COM
I20260807 06:38:36.606415 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.599222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55386 (local address 127.11.114.253:39293)
0807 06:38:36.599269 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.599270 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.599408 (+ 138us) server_negotiation.cc:407] Connection header received
0807 06:38:36.599484 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.599485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.599493 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.599534 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.600034 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.600464 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.601595 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.601787 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.601833 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.603886 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.603888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.603888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.603895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.605167 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.605611 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.605612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.605612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.605671 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.605879 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.605880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.605881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.605988 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.606063 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.606250 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.606325 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 06:38:36.606442 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.599141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.0.0.1:55386)
0807 06:38:36.599288 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.599338 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.599413 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.599548 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.599551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.599898 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.599956 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.599958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.600541 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.600542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.601469 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.601472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.601856 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.601857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.601957 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.602353 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.602359 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.603785 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.605275 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.605282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.605285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.605484 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.605749 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.605750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.605750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.605814 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.606057 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.606064 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.606115 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.606215 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.606329 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260807 06:38:36.606928 20217 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.114.253:39293: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b"
member_type: NON_VOTER
last_known_addr {
host: "127.11.114.253"
port: 39293
}
attrs {
promote: true
}
}
dest_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d"
cas_config_opid_index: -1
I20260807 06:38:36.607280 20217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } }
I20260807 06:38:36.607754 21044 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bbd1d4d07d1e4bd885041290cffa1e1b (127.11.114.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } }
I20260807 06:38:36.608047 20969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:36.608619 21044 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.114.253:39293
I20260807 06:38:36.608916 20902 master.cc:537] Master@127.11.114.253:39293 shutting down...
I20260807 06:38:36.608961 20969 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Refusing update from remote peer 27284d9f9c8e4ff4a0c93a466cc3af4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 06:38:36.608920 21045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } }
I20260807 06:38:36.609004 21045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:36.609256 21045 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:38:36.609993 20198 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 06:38:36.611007 20198 consensus_peers.cc:487] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d -> Peer bbd1d4d07d1e4bd885041290cffa1e1b (127.11.114.253:39293): Couldn't send request to peer bbd1d4d07d1e4bd885041290cffa1e1b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 06:38:36.611835 20902 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:36.611931 20902 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:36.611980 20902 tablet_replica.cc:330] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: stopping tablet replica
I20260807 06:38:36.625262 20902 master.cc:552] Master@127.11.114.253:39293 shutdown complete.
I20260807 06:38:36.625319 20902 master_runner.cc:305] Clearing existing system tablet
I20260807 06:38:36.625458 20902 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.625509 20902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.626288 20902 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:36.626816 21053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.626966 20902 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.627027 20902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.627053 20902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
uuid: "bbd1d4d07d1e4bd885041290cffa1e1b"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "533ef7f1def0d4e0e9cb744692a50d90"
server_key_iv: "4d25ec96e865d29d99f38e0279d52951"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.627943 20902 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:36.629340 20902 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 06:38:36.629434 20902 log.cc:1196] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 06:38:36.629783 20902 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Deleting consensus metadata
I20260807 06:38:36.629914 20902 master_runner.cc:315] Copying system tablet from 127.11.114.254:33755
I20260807 06:38:36.630548 20902 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: tablet copy: Beginning tablet copy session from remote peer at address 127.11.114.254:33755
I20260807 06:38:36.637200 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.630958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48860 (local address 127.11.114.254:33755)
0807 06:38:36.631010 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.631012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.631086 (+ 74us) server_negotiation.cc:407] Connection header received
0807 06:38:36.631165 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.631166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.631177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.631235 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.631785 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.632570 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.633271 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.633606 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.633689 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.634625 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.634627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.634627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.634634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.636136 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.636484 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.636486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.636486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.636549 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.636724 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.636725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.636726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.636856 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.636923 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.637030 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.637106 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:36.637207 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.630891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:48860)
0807 06:38:36.631021 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.631031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.631098 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.631254 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.631254 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.631415 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.631699 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.631701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.632656 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.632657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.633196 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.633198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.633684 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.633685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.633740 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.634237 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.634242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.634537 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.636219 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.636220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.636221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.636412 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.636613 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.636614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.636614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.636666 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.636917 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.636918 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.636954 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.637004 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.637117 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":94,"threads_started":1}
I20260807 06:38:36.637925 20241 tablet_copy_service.cc:140] P 27284d9f9c8e4ff4a0c93a466cc3af4d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bbd1d4d07d1e4bd885041290cffa1e1b ({username='oryx', principal='oryx/127.11.114.253@KRBTEST.COM'} at 127.0.0.1:48860)
I20260807 06:38:36.638063 20241 tablet_copy_service.cc:161] P 27284d9f9c8e4ff4a0c93a466cc3af4d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bbd1d4d07d1e4bd885041290cffa1e1b at {username='oryx', principal='oryx/127.11.114.253@KRBTEST.COM'} at 127.0.0.1:48860: session id = bbd1d4d07d1e4bd885041290cffa1e1b-00000000000000000000000000000000
I20260807 06:38:36.638929 20241 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 06:38:36.639503 20902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 06:38:36.640476 20902 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: tablet copy: Starting download of 0 data blocks...
I20260807 06:38:36.640640 20902 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: tablet copy: Starting download of 1 WAL segments...
I20260807 06:38:36.641649 20902 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 06:38:36.642445 20241 tablet_copy_service.cc:342] P 27284d9f9c8e4ff4a0c93a466cc3af4d: Request end of tablet copy session bbd1d4d07d1e4bd885041290cffa1e1b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.114.253@KRBTEST.COM'} at 127.0.0.1:48860
I20260807 06:38:36.642504 20241 tablet_copy_service.cc:433] P 27284d9f9c8e4ff4a0c93a466cc3af4d: ending tablet copy session bbd1d4d07d1e4bd885041290cffa1e1b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bbd1d4d07d1e4bd885041290cffa1e1b
I20260807 06:38:36.645088 20902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:36.646266 21065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.646277 21066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.646426 20902 server_base.cc:1034] running on GCE node
W20260807 06:38:36.646286 21072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.646607 20902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:36.646770 20902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:36.647888 20902 hybrid_clock.cc:648] HybridClock initialized: now 1786084716647880 us; error 25 us; skew 500 ppm
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:36.649196 20902 init.cc:377] Logged in from keytab as oryx/127.11.114.253@KRBTEST.COM (short username oryx)
I20260807 06:38:36.649477 20902 webserver.cc:460] Webserver started at http://127.11.114.253:36613/ using document root <none> and password file <none>
I20260807 06:38:36.649632 20902 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.649681 20902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.650280 20902 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:36.650820 21081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.650957 20902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:36.651026 20902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
uuid: "bbd1d4d07d1e4bd885041290cffa1e1b"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "533ef7f1def0d4e0e9cb744692a50d90"
server_key_iv: "4d25ec96e865d29d99f38e0279d52951"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.651086 20902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.659539 20902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:36.661036 20902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:36.664372 20902 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.253:39293
I20260807 06:38:36.664772 20902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
I20260807 06:38:36.664799 21156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.253:39293 every 8 connection(s)
I20260807 06:38:36.665786 21157 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:38:36.666437 21157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Bootstrap starting.
I20260807 06:38:36.669687 21157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:38:36.669932 21157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: Bootstrap complete.
I20260807 06:38:36.670127 21157 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } }
I20260807 06:38:36.670215 21157 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bbd1d4d07d1e4bd885041290cffa1e1b, State: Initialized, Role: LEARNER
I20260807 06:38:36.670253 21157 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } }
I20260807 06:38:36.670433 21157 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:36.670430 21161 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } }
I20260807 06:38:36.670567 21161 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: This master's current role is: LEARNER
I20260807 06:38:36.671916 21177 catalog_manager.cc:1259] Loaded cluster ID: 87ae0a6e997546f9bdd84ff1a1d87d4c
I20260807 06:38:36.672446 21177 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: loading cluster ID for follower catalog manager: success
I20260807 06:38:36.673270 21177 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: acquiring CA information for follower catalog manager: success
I20260807 06:38:36.673616 21177 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:38:36.687337 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.680328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.11.114.253:39293)
0807 06:38:36.680461 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.680464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.680475 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:38:36.680502 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.680503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.680514 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.680608 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.681128 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.682213 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.682919 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.683217 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.683291 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.684517 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.684519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.684519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.684526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.686177 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.686520 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.686521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.686522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.686603 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.686800 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.686801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.686801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.686941 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.687046 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.687139 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.687228 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
WARNING: no policy specified for oryx/127.11.114.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.252.keytab.
Entry for principal oryx/127.11.114.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.252.keytab.
I20260807 06:38:36.714633 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.252:33895
--webserver_interface=127.11.114.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.252.keytab
--principal=oryx/127.11.114.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:33755,127.11.114.253:39293,127.11.114.252:33895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:36.797509 21183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:36.797780 21183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:36.797817 21183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:36.798939 21183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:36.799005 21183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:36.799029 21183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:36.799062 21183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:36.799105 21183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:36.800581 21183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:33755,127.11.114.253:39293,127.11.114.252:33895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.252.keytab
--principal=oryx/127.11.114.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.252:33895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.114.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:36.800858 21183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:36.801074 21183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:36.806285 21188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.806416 21189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.806332 21193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.806567 21183 server_base.cc:1034] running on GCE node
I20260807 06:38:36.806805 21183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:36.807071 21183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:36.808223 21183 hybrid_clock.cc:648] HybridClock initialized: now 1786084716808208 us; error 29 us; skew 500 ppm
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:36.810950 21183 init.cc:377] Logged in from keytab as oryx/127.11.114.252@KRBTEST.COM (short username oryx)
I20260807 06:38:36.811441 21183 webserver.cc:460] Webserver started at http://127.11.114.252:45231/ using document root <none> and password file <none>
I20260807 06:38:36.811687 21183 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.811738 21183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.811924 21183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:36.813102 21183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data/instance:
uuid: "ee5a25e82eca47e69d975cccd2148e37"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "86c65ec145a914462a7bd98324b5ca63"
server_key_iv: "03bd70bba3829428832bf55dc6919b7f"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.813495 21183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal/instance:
uuid: "ee5a25e82eca47e69d975cccd2148e37"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "86c65ec145a914462a7bd98324b5ca63"
server_key_iv: "03bd70bba3829428832bf55dc6919b7f"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.815145 21183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:36.816006 21203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.816226 21183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:36.816336 21183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
uuid: "ee5a25e82eca47e69d975cccd2148e37"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "86c65ec145a914462a7bd98324b5ca63"
server_key_iv: "03bd70bba3829428832bf55dc6919b7f"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.816416 21183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.835295 21183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:36.837455 21183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:36.837693 21183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:36.843276 21183 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.252:33895
I20260807 06:38:36.843310 21278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.252:33895 every 8 connection(s)
I20260807 06:38:36.843712 21183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
I20260807 06:38:36.844695 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:36.846352 21279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:38:36.850251 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 21183
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.252@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.114.252@KRBTEST.COM
I20260807 06:38:36.856927 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.847215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48862 (local address 127.11.114.254:33755)
0807 06:38:36.847320 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.847322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.847436 (+ 114us) server_negotiation.cc:407] Connection header received
0807 06:38:36.847966 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.847967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.847980 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.848053 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.848644 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.849158 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.850211 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.850560 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.850650 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.853643 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.853645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.853645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.853653 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.855395 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.855880 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.855881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.855881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.855938 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.856223 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.856225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.856225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.856384 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.856502 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.856720 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.856815 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 06:38:36.857023 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.847113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:48862)
0807 06:38:36.847310 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.847346 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.847869 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.848122 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.848125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.848369 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.848562 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.848566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.849240 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.849241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.850097 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.850099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.850710 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.850712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.850881 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.851543 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.851553 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.853436 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.855481 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.855489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.855491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.855801 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.856010 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.856011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.856012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.856138 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.856464 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.856469 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.856583 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.856667 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.856783 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260807 06:38:36.857707 21279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:38:36.860356 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.850888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.11.114.252:33895)
0807 06:38:36.851099 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.851104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.851129 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:38:36.851160 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.851163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.851197 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.851281 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.852097 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.853303 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.854121 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.854422 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.854502 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.857354 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.857358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.857359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.857366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.859086 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.859536 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.859538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.859538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.859594 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.859816 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.859817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.859817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.859940 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.860046 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.860183 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.860267 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":121,"threads_started":1}
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.252@KRBTEST.COM for oryx/127.11.114.253@KRBTEST.COM
I20260807 06:38:36.864848 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.857996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.0.0.1:55408)
0807 06:38:36.858055 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.858066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.858108 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.858253 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.858254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.858404 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.858457 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.858459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.859234 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.859235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.860109 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.860112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.860721 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.860722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.860779 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.861099 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.861103 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.862322 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.863923 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.863925 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.863925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.864168 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.864355 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.864356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.864356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.864418 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.864622 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.864623 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.864658 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.864717 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.864788 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 06:38:36.864858 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.858075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55408 (local address 127.11.114.253:39293)
0807 06:38:36.858131 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.858134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.858146 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:38:36.858177 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.858177 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.858191 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.858241 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.858558 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.859097 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.860216 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.860581 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.860629 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.862403 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.862404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.862404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.862411 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.863853 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.864242 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.864243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.864243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.864292 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.864472 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.864473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.864473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.864568 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.864615 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.864736 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.864800 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:36.865286 21279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.252@KRBTEST.COM for oryx/127.11.114.252@KRBTEST.COM
I20260807 06:38:36.871939 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.865512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.0.0.1:45390)
0807 06:38:36.865620 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.865633 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.865681 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.865815 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.865816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.865953 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.865978 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.865979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.866672 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.866673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.867185 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.867186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.867631 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.867632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.867687 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.867985 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.867990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.869095 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.870689 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.870690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.870690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.870966 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.871190 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.871191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.871192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.871254 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.871585 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.871586 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.871627 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.871722 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.871824 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 06:38:36.872082 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.865643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45390 (local address 127.11.114.252:33895)
0807 06:38:36.865675 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.865677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.865685 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:36.865743 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.865744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.865755 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.865801 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.866076 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.866571 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.867288 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.867534 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.867630 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.869186 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.869187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.869187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.869194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.870590 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.871051 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.871052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.871053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.871117 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.871331 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.871333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.871333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.871487 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.871592 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.871747 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.871898 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":17}
I20260807 06:38:36.873456 21279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Bootstrap starting.
I20260807 06:38:36.874330 21279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:36.874558 21279 log.cc:824] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:36.875257 21279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: No bootstrap required, opened a new log
I20260807 06:38:36.876621 21279 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } }
I20260807 06:38:36.876756 21279 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:36.876780 21279 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5a25e82eca47e69d975cccd2148e37, State: Initialized, Role: FOLLOWER
I20260807 06:38:36.876902 21279 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } }
I20260807 06:38:36.877148 21286 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } } }
I20260807 06:38:36.877204 21279 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:36.877256 21286 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:38:36.879123 21301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:38:36.879189 21301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:36.881184 21183 master_runner.cc:428] Detected that this master ee5a25e82eca47e69d975cccd2148e37 is joining an existing cluster
I20260807 06:38:36.881237 21183 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.114.252:33895
I20260807 06:38:36.893328 21307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.885942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:48864)
0807 06:38:36.886247 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.886256 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.886412 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.886592 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.886593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.886835 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.886882 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.886884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.887921 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.887922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.888896 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.888898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.889420 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.889420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.889482 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.889952 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.889957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.890290 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.892259 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.892260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.892261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.892442 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.892673 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.892674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.892674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.892726 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.893096 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.893097 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.893133 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.893182 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.893256 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":32,"thread_start_us":80,"threads_started":1}
I20260807 06:38:36.893422 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.885797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.11.114.254:33755)
0807 06:38:36.885905 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.885907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.886388 (+ 481us) server_negotiation.cc:407] Connection header received
0807 06:38:36.886486 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.886486 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.886497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.886529 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.886976 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.887825 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.888989 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.889286 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.889325 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.890395 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.890397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.890397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.890405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.892178 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.892519 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.892520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.892521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.892600 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.892801 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.892802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.892803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.893018 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.893090 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.893235 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.893320 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084715, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.254@KRBTEST.COM for oryx/127.11.114.252@KRBTEST.COM
I20260807 06:38:36.911960 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 20310
I20260807 06:38:36.931205 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.895480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.0.0.1:45400)
0807 06:38:36.895533 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.895543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.895578 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.895856 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.895857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.896000 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.896031 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.896032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.896664 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.896665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.897154 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.897155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.897633 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.897634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.897703 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.898131 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.898135 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.899570 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.929947 (+ 30377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.929948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.929949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.930205 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.930431 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.930432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.930432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.930495 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.930805 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.930806 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.930861 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.930909 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.931042 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 06:38:36.932300 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.895567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45400 (local address 127.11.114.252:33895)
0807 06:38:36.895716 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.895717 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.895723 (+ 6us) server_negotiation.cc:407] Connection header received
0807 06:38:36.895762 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.895762 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.895770 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.895839 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.896113 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.896583 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.897230 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.897446 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.897646 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.899686 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.899687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.899687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.899694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.929834 (+ 30140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.930293 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.930294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.930294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.930361 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.930561 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.930562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.930562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.930714 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.931404 (+ 690us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.931452 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.932161 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260807 06:38:36.932983 20217 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.114.252:33895: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37"
member_type: NON_VOTER
last_known_addr {
host: "127.11.114.252"
port: 33895
}
attrs {
promote: true
}
}
dest_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d"
cas_config_opid_index: 5
I20260807 06:38:36.933246 20217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } }
I20260807 06:38:36.933863 21045 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.934147 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:44647
--local_ip_for_outbound_sockets=127.11.114.193
--tserver_master_addrs=127.11.114.254:33755
--webserver_port=35893
--webserver_interface=127.11.114.193
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab
--principal=oryx/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293,127.11.114.252:33895 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:38:36.934989 21045 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.114.252:33895
I20260807 06:38:36.935066 21045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.935144 21045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:36.935174 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.885853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.11.114.252:33895)
0807 06:38:36.885921 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.885923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.886165 (+ 242us) server_negotiation.cc:407] Connection header received
0807 06:38:36.886264 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.886264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.886273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.886303 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.886794 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.887317 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.889054 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.889358 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.889400 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.932042 (+ 42642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.932044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.932044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.932051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.933960 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.934387 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.934388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.934388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.934445 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.934648 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.934649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.934650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.934774 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.934814 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.934995 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.935076 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:38:36.935432 21183 master.cc:537] Master@127.11.114.252:33895 shutting down...
I20260807 06:38:36.935459 21308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.885812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.0.0.1:45398)
0807 06:38:36.886095 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.886107 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.886190 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.886358 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.886359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.886603 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.886722 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.886724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.887946 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.887948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.888949 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.888952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.889479 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.889480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.889587 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.931395 (+ 41808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.931400 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.931879 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.934053 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.934053 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.934054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.934300 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.934514 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.934514 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.934515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.934567 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.934855 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.934856 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.934893 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.935250 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.935352 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":44,"thread_start_us":90,"threads_started":1}
W20260807 06:38:36.937397 20197 consensus_peers.cc:487] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d -> Peer ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252:33895): Couldn't send request to peer ee5a25e82eca47e69d975cccd2148e37. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 06:38:36.937606 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.886109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.11.114.253:39293)
0807 06:38:36.886153 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.886155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.886307 (+ 152us) server_negotiation.cc:407] Connection header received
0807 06:38:36.886407 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.886408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.886420 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.886451 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.887554 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.888137 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.889093 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.889393 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.889433 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.927756 (+ 38323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.927759 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.927759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.927772 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.930784 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.936646 (+ 5862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.936647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.936648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.936713 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.936908 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.936909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.936910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.937029 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.937093 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.937372 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.937466 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:38:36.938009 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.886090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.0.0.1:55418)
0807 06:38:36.886246 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.886256 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.886331 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.886520 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.886521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.887429 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.887475 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.887476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.888238 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.888239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.889012 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.889014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.889503 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.889504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.889589 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.926486 (+ 36897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.926494 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.927114 (+ 620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.931934 (+ 4820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.931935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.931936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.936559 (+ 4623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.936785 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.936786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.936786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.936845 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.937187 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.937188 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.937246 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.937786 (+ 540us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.937894 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260807 06:38:36.940459 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.934011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.0.0.1:55420)
0807 06:38:36.934240 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.934253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.934300 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.936243 (+ 1943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.936244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.936247 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:36.936298 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.936300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.937471 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.937473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.938869 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.938873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.940201 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.940202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.940279 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.940307 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.940342 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.940382 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260807 06:38:36.940943 21183 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:36.941006 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.934140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55420 (local address 127.11.114.253:39293)
0807 06:38:36.936045 (+ 1905us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.936048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.936056 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:36.936083 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.936084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.936112 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.936365 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:36.936402 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.937362 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.939253 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.940084 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.940772 (+ 688us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.940859 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.940881 (+ 22us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.940930 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1883,"thread_start_us":57,"threads_started":1}
I20260807 06:38:36.941040 21183 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:36.941097 21183 tablet_replica.cc:330] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: stopping tablet replica
I20260807 06:38:36.941210 21121 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 LEARNER]: Refusing update from remote peer 27284d9f9c8e4ff4a0c93a466cc3af4d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 06:38:36.941594 21312 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 06:38:36.942293 21121 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.943775 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.943601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.0.0.1:45402)
0807 06:38:36.943696 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.943742 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:33895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 06:38:36.943935 21161 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27284d9f9c8e4ff4a0c93a466cc3af4d. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.944057 21161 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: This master's current role is: LEARNER
I20260807 06:38:36.944100 21161 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: NON_VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: true } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.944145 21161 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: This master's current role is: LEARNER
I20260807 06:38:36.945185 21044 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: attempting to promote NON_VOTER bbd1d4d07d1e4bd885041290cffa1e1b to VOTER
I20260807 06:38:36.945399 21044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } }
I20260807 06:38:36.945875 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.945761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.0.0.1:45418)
0807 06:38:36.945803 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.945843 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:33895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 06:38:36.945871 21121 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 LEARNER]: Refusing update from remote peer 27284d9f9c8e4ff4a0c93a466cc3af4d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 06:38:36.946089 21045 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 06:38:36.946162 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.946088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.0.0.1:45422)
0807 06:38:36.946120 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.946140 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:33895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 06:38:36.946210 20197 consensus_peers.cc:487] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d -> Peer ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252:33895): Couldn't send request to peer ee5a25e82eca47e69d975cccd2148e37. Status: Network error: Client connection negotiation failed: client connection to 127.11.114.252:33895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 06:38:36.946743 21045 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bbd1d4d07d1e4bd885041290cffa1e1b (127.11.114.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.947702 21045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.947779 21045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:36.947832 21045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.947876 21045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:36.948246 21121 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bbd1d4d07d1e4bd885041290cffa1e1b (127.11.114.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.949401 21318 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:36.949719 21318 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:38:36.956211 21183 master.cc:552] Master@127.11.114.252:33895 shutdown complete.
I20260807 06:38:36.956276 21183 master_runner.cc:305] Clearing existing system tablet
I20260807 06:38:36.956416 21183 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.956466 21183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.957250 21183 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:36.957862 21321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.957983 21183 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.958033 21183 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.958058 21183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
uuid: "ee5a25e82eca47e69d975cccd2148e37"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "86c65ec145a914462a7bd98324b5ca63"
server_key_iv: "03bd70bba3829428832bf55dc6919b7f"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.958927 21183 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:36.960640 21183 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 06:38:36.960726 21183 log.cc:1196] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 06:38:36.960958 21183 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Deleting consensus metadata
I20260807 06:38:36.961112 21183 master_runner.cc:315] Copying system tablet from 127.11.114.254:33755
I20260807 06:38:36.961866 21183 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: tablet copy: Beginning tablet copy session from remote peer at address 127.11.114.254:33755
I20260807 06:38:36.970201 21331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.962178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:48870)
0807 06:38:36.962314 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:36.962325 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:36.962393 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:36.962563 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:36.962564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:36.962767 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:36.962841 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.962843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.963946 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.963948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.964813 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:36.964816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.965384 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.965385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:36.965457 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.966463 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:36.966469 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:36.966897 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:36.968915 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.968916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.968917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.969144 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.969379 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:36.969380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:36.969381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:36.969451 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:36.969855 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:36.969856 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:36.969908 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 06:38:36.969985 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:38:36.970094 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260807 06:38:36.970228 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:36.962272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.11.114.254:33755)
0807 06:38:36.962325 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:36.962327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:36.962381 (+ 54us) server_negotiation.cc:407] Connection header received
0807 06:38:36.962450 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:36.962451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:36.962464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:36.962538 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:36.962918 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.963863 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.964912 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:36.965290 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:36.965374 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:36.966997 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:36.966999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:36.966999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:36.967008 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:36.968820 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.969230 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.969232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.969232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.969304 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:36.969523 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:36.969524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:36.969525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:36.969774 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:36.969856 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:36.970033 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:38:36.970123 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 06:38:36.970928 20241 tablet_copy_service.cc:140] P 27284d9f9c8e4ff4a0c93a466cc3af4d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ee5a25e82eca47e69d975cccd2148e37 ({username='oryx', principal='oryx/127.11.114.252@KRBTEST.COM'} at 127.0.0.1:48870)
I20260807 06:38:36.971002 20241 tablet_copy_service.cc:161] P 27284d9f9c8e4ff4a0c93a466cc3af4d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ee5a25e82eca47e69d975cccd2148e37 at {username='oryx', principal='oryx/127.11.114.252@KRBTEST.COM'} at 127.0.0.1:48870: session id = ee5a25e82eca47e69d975cccd2148e37-00000000000000000000000000000000
I20260807 06:38:36.971688 20241 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 06:38:36.972219 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 06:38:36.973626 21183 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: tablet copy: Starting download of 0 data blocks...
I20260807 06:38:36.973785 21183 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: tablet copy: Starting download of 1 WAL segments...
I20260807 06:38:36.974609 21183 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 06:38:36.975560 20241 tablet_copy_service.cc:342] P 27284d9f9c8e4ff4a0c93a466cc3af4d: Request end of tablet copy session ee5a25e82eca47e69d975cccd2148e37-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.114.252@KRBTEST.COM'} at 127.0.0.1:48870
I20260807 06:38:36.975629 20241 tablet_copy_service.cc:433] P 27284d9f9c8e4ff4a0c93a466cc3af4d: ending tablet copy session ee5a25e82eca47e69d975cccd2148e37-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ee5a25e82eca47e69d975cccd2148e37
I20260807 06:38:36.978650 21183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:36.979820 21338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.979820 21333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:36.980122 21335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:36.980472 21183 server_base.cc:1034] running on GCE node
I20260807 06:38:36.980571 21183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:36.980700 21183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:36.981819 21183 hybrid_clock.cc:648] HybridClock initialized: now 1786084716981810 us; error 27 us; skew 500 ppm
Aug 07 06:38:36 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084716, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:36.983402 21183 init.cc:377] Logged in from keytab as oryx/127.11.114.252@KRBTEST.COM (short username oryx)
I20260807 06:38:36.983668 21183 webserver.cc:460] Webserver started at http://127.11.114.252:33061/ using document root <none> and password file <none>
I20260807 06:38:36.983783 21183 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:36.983816 21183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:36.984562 21183 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:38:36.985169 21347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:36.985446 21183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:38:36.985502 21183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
uuid: "ee5a25e82eca47e69d975cccd2148e37"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "86c65ec145a914462a7bd98324b5ca63"
server_key_iv: "03bd70bba3829428832bf55dc6919b7f"
server_key_version: "encryptionkey@0"
I20260807 06:38:36.985610 21183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:36.993062 21183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:36.995064 21183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:36.998119 21183 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.252:33895
I20260807 06:38:36.998507 21183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
I20260807 06:38:36.999344 21423 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:38:36.999820 21423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Bootstrap starting.
I20260807 06:38:37.001065 21419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.252:33895 every 8 connection(s)
I20260807 06:38:37.002786 21423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:38:37.002993 21423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Bootstrap complete.
I20260807 06:38:37.003168 21423 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } }
I20260807 06:38:37.003271 21423 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ee5a25e82eca47e69d975cccd2148e37, State: Initialized, Role: LEARNER
I20260807 06:38:37.003305 21423 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } }
I20260807 06:38:37.003466 21423 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:37.004752 21427 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:37.004902 21427 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:38:37.010003 21442 catalog_manager.cc:1259] Loaded cluster ID: 87ae0a6e997546f9bdd84ff1a1d87d4c
I20260807 06:38:37.010075 21442 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: loading cluster ID for follower catalog manager: success
I20260807 06:38:37.011364 21442 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: acquiring CA information for follower catalog manager: success
I20260807 06:38:37.011726 21442 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 06:38:37.039491 21315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:37.039709 21315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:37.039757 21315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:37.040897 21315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:37.040967 21315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:37.041014 21315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:37.042582 21315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab
--principal=oryx/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:44647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=35893
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293,127.11.114.252:33895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:37.042836 21315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:37.043114 21315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:37.048235 21450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:37.048229 21447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:37.048422 21448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:37.048676 21315 server_base.cc:1034] running on GCE node
I20260807 06:38:37.048907 21315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:37.049194 21315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:37.050346 21315 hybrid_clock.cc:648] HybridClock initialized: now 1786084717050329 us; error 33 us; skew 500 ppm
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:37.053264 21315 init.cc:377] Logged in from keytab as oryx/127.11.114.193@KRBTEST.COM (short username oryx)
I20260807 06:38:37.053853 21315 webserver.cc:460] Webserver started at http://127.11.114.193:35893/ using document root <none> and password file <none>
I20260807 06:38:37.054147 21315 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:37.054227 21315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:37.056043 21315 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:37.056794 21459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.057127 21315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:37.057231 21315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "110dec602478421ba7f0fd9d6806ab03"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "f14911dfb54c62c1d01545ab1f48eb80"
server_key_iv: "8801b3c84f9746d5326b5803ff44fae9"
server_key_version: "encryptionkey@0"
I20260807 06:38:37.057730 21315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:37.091928 21315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:37.094307 21315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:37.094475 21315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:37.094718 21315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:37.095062 21315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:37.095114 21315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.095157 21315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:37.095194 21315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.101415 21315 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:44647
I20260807 06:38:37.101493 21626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:44647 every 8 connection(s)
I20260807 06:38:37.101902 21315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:37.103976 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 21315
I20260807 06:38:37.104194 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 20512
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.252@KRBTEST.COM
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.253@KRBTEST.COM
I20260807 06:38:37.115422 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:37155
--local_ip_for_outbound_sockets=127.11.114.194
--tserver_master_addrs=127.11.114.254:33755
--webserver_port=41923
--webserver_interface=127.11.114.194
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab
--principal=oryx/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293,127.11.114.252:33895 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:38:37.115985 21632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.102667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.11.114.193:58969)
0807 06:38:37.102916 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.102957 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.104365 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.104742 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.104745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.105080 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.105331 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.105335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.106401 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.106402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.108100 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.108103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.108759 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.108761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.108936 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.109586 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.109596 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.111723 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.114125 (+ 2402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.114131 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.114134 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.114458 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.114677 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.114679 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.114679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.114813 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.115139 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.115146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.115318 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.115441 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.115616 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":151,"threads_started":1}
I20260807 06:38:37.117300 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.102662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:58969 (local address 127.11.114.252:33895)
0807 06:38:37.103259 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.103261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.103270 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:37.104563 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.104564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.104579 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.104632 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.105422 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.106316 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.108205 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.108614 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.108685 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.111836 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.111837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.111838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.111846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.114028 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.114545 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.114546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.114547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.114610 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.114887 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.114888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.114888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.115042 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.115108 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.117059 (+ 1951us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.117171 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":69,"threads_started":1}
I20260807 06:38:37.117975 21627 heartbeater.cc:347] Connected to a master server at 127.11.114.252:33895
I20260807 06:38:37.118079 21627 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.118295 21627 heartbeater.cc:511] Master 127.11.114.252:33895 requested a full tablet report, sending...
I20260807 06:38:37.118536 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.103071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:43973 (local address 127.11.114.253:39293)
0807 06:38:37.103118 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.103121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.103256 (+ 135us) server_negotiation.cc:407] Connection header received
0807 06:38:37.110505 (+ 7249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.110507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.110519 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.110569 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.110888 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.111468 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.112378 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.112587 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.112674 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.114908 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.114909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.114909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.114917 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.116947 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.117517 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.117518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.117519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.117617 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.117838 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.117839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.117839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.117993 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.118040 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.118301 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.118409 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:38:37.118875 21368 ts_manager.cc:194] Registered new tserver with Master: 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647)
I20260807 06:38:37.118945 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.102980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.11.114.193:43973)
0807 06:38:37.103146 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.103156 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.110407 (+ 7251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.110571 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.110572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.110751 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.110811 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.110813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.111564 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.111564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.112296 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.112298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.112674 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.112675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.112756 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.113179 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.113184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.114808 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.117170 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.117171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.117172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.117411 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.117703 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.117704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.117704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.117765 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.118104 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.118105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.118153 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.118655 (+ 502us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.118816 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":67,"threads_started":1}
I20260807 06:38:37.119401 21629 heartbeater.cc:347] Connected to a master server at 127.11.114.253:39293
I20260807 06:38:37.119459 21629 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.119529 21629 heartbeater.cc:511] Master 127.11.114.253:39293 requested a full tablet report, sending...
I20260807 06:38:37.119954 21108 ts_manager.cc:194] Registered new tserver with Master: 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647)
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:37.123519 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.102724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:58849 (local address 127.11.114.254:33755)
0807 06:38:37.102831 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.102833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.103182 (+ 349us) server_negotiation.cc:407] Connection header received
0807 06:38:37.114100 (+ 10918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.114102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.114118 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.114191 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.114768 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.116319 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.117518 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.117909 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.117956 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.120507 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.120509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.120509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.120517 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.122246 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.122649 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.122650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.122651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.122718 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.122941 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.122942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.122943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.123068 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.123153 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.123336 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.123417 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 06:38:37.123543 21633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.102667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:58849)
0807 06:38:37.103114 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.103125 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.104485 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.114332 (+ 9847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.114334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.114564 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.114656 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.114658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.116431 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.116432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.117399 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.117402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.118047 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.118048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.118129 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.118626 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.118632 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.120398 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.122359 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.122360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.122361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.122556 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.122799 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.122800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.122800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.122870 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.123160 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.123161 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.123208 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.123302 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.123419 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":4,"thread_start_us":162,"threads_started":1}
I20260807 06:38:37.124163 21628 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33755
I20260807 06:38:37.124214 21628 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.124307 21628 heartbeater.cc:511] Master 127.11.114.254:33755 requested a full tablet report, sending...
I20260807 06:38:37.124644 20217 ts_manager.cc:194] Re-registered known tserver with Master: 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647)
I20260807 06:38:37.125198 20217 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:58849
W20260807 06:38:37.202538 21637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:37.202757 21637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:37.202782 21637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:37.204186 21637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:37.204231 21637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:37.204272 21637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:37.205866 21637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab
--principal=oryx/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:37155
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=41923
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293,127.11.114.252:33895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:37.206125 21637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:37.206391 21637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:37.211661 21644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:37.211701 21646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:37.211752 21637 server_base.cc:1034] running on GCE node
W20260807 06:38:37.211872 21650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:37.212126 21637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:37.212329 21637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:37.213480 21637 hybrid_clock.cc:648] HybridClock initialized: now 1786084717213452 us; error 50 us; skew 500 ppm
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:37.216153 21637 init.cc:377] Logged in from keytab as oryx/127.11.114.194@KRBTEST.COM (short username oryx)
I20260807 06:38:37.216675 21637 webserver.cc:460] Webserver started at http://127.11.114.194:41923/ using document root <none> and password file <none>
I20260807 06:38:37.216917 21637 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:37.216977 21637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:37.218415 21637 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:37.219045 21661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.219390 21637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:37.219502 21637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "31ddfc1398ad41d98c620c704c2e2ad4"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "41f6d445c42515691c13c1675c652b80"
server_key_iv: "62cf7f5d3c254bbf9743c120bf95a41b"
server_key_version: "encryptionkey@0"
I20260807 06:38:37.219842 21637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:37.262365 21637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:37.264683 21637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:37.264859 21637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:37.265130 21637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:37.265470 21637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:37.265524 21637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.265581 21637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:37.265630 21637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.272650 21637 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:37155
I20260807 06:38:37.272845 21810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:37155 every 8 connection(s)
I20260807 06:38:37.273138 21637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.252@KRBTEST.COM
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.253@KRBTEST.COM
I20260807 06:38:37.283474 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 21637
I20260807 06:38:37.283643 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 20712
I20260807 06:38:37.288877 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.273976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:43163 (local address 127.11.114.252:33895)
0807 06:38:37.274042 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.274044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.274296 (+ 252us) server_negotiation.cc:407] Connection header received
0807 06:38:37.275128 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.275129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.275144 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.275218 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.276769 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.277605 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.279144 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.279455 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.279506 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.282541 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.282542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.282543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.282550 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.284417 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.287726 (+ 3309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.287728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.287728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.287795 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.288064 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.288065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.288066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.288231 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.288299 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.288679 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.288783 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:38:37.289212 21818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.273951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.11.114.194:43163)
0807 06:38:37.274152 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.274196 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.275009 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.275319 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.275324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.276273 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.276607 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.276614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.277722 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.277723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.279023 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.279026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.279608 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.279609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.279762 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.280662 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.280667 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.282451 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.287275 (+ 4824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.287284 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.287287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.287631 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.287866 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.287867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.287867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.288006 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.288326 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.288340 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.288527 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.288645 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.288831 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260807 06:38:37.289224 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.277143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:60725 (local address 127.11.114.254:33755)
0807 06:38:37.277187 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.277188 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.277195 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:38:37.277219 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.277219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.277229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.277282 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.277948 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.278715 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.279741 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.280042 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.280085 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.283032 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.283033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.283034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.283042 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.287854 (+ 4812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.288280 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.288281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.288281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.288347 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.288554 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.288555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.288555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.288713 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.288781 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.288970 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.289066 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:37.289942 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.276970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:60725)
0807 06:38:37.277073 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.277083 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.277127 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.277314 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.277315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.277741 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.277839 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.277841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.278814 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.278815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.279649 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.279651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.280168 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.280169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.280217 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.280650 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.280660 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.282942 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.287952 (+ 5010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.287953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.287954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.288183 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.288414 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.288415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.288415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.288481 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.288810 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.288811 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.288858 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.289786 (+ 928us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.289873 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":64,"threads_started":1}
I20260807 06:38:37.290328 21814 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33755
I20260807 06:38:37.290431 21814 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.290614 21814 heartbeater.cc:511] Master 127.11.114.254:33755 requested a full tablet report, sending...
I20260807 06:38:37.291049 20217 ts_manager.cc:194] Re-registered known tserver with Master: 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155)
I20260807 06:38:37.291630 20217 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:60725
I20260807 06:38:37.294376 21812 heartbeater.cc:347] Connected to a master server at 127.11.114.252:33895
I20260807 06:38:37.294441 21812 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.294521 21812 heartbeater.cc:511] Master 127.11.114.252:33895 requested a full tablet report, sending...
I20260807 06:38:37.294538 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:38629
--local_ip_for_outbound_sockets=127.11.114.195
--tserver_master_addrs=127.11.114.254:33755
--webserver_port=38199
--webserver_interface=127.11.114.195
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab
--principal=oryx/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293,127.11.114.252:33895 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:38:37.294853 21368 ts_manager.cc:194] Registered new tserver with Master: 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155)
I20260807 06:38:37.297698 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.276611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.11.114.194:41031)
0807 06:38:37.276886 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.276897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.276944 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.277555 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.277556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.277788 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.277932 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.277934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.279257 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.279258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.280041 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.280043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.280565 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.280566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.280649 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.281061 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.281065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.294549 (+ 13484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.296548 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.296549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.296549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.296808 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.297004 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.297005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.297006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.297086 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.297327 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.297328 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.297368 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.297428 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.297601 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":10,"thread_start_us":76,"threads_started":1}
I20260807 06:38:37.297782 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.276766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:41031 (local address 127.11.114.253:39293)
0807 06:38:37.277381 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.277383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.277392 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:37.277418 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.277419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.277430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.277485 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.278054 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.279145 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.280127 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.280483 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.280554 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.294840 (+ 14286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.294843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.294843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.294853 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.296458 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.296886 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.296886 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.296887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.296939 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.297152 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.297153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.297153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.297261 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.297312 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.297490 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.297641 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589}
I20260807 06:38:37.298292 21816 heartbeater.cc:347] Connected to a master server at 127.11.114.253:39293
I20260807 06:38:37.298341 21816 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.298381 21816 heartbeater.cc:511] Master 127.11.114.253:39293 requested a full tablet report, sending...
I20260807 06:38:37.298738 21108 ts_manager.cc:194] Registered new tserver with Master: 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155)
W20260807 06:38:37.380533 21821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:37.380751 21821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:37.380775 21821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:37.381922 21821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:37.381959 21821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:37.381985 21821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:37.383397 21821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab
--principal=oryx/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:38629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=38199
--tserver_master_addrs=127.11.114.254:33755,127.11.114.253:39293,127.11.114.252:33895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:37.383584 21821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:37.383769 21821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:37.388921 21833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:37.389037 21821 server_base.cc:1034] running on GCE node
W20260807 06:38:37.388921 21829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:37.389101 21830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:37.389374 21821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:37.389689 21821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:37.390870 21821 hybrid_clock.cc:648] HybridClock initialized: now 1786084717390854 us; error 32 us; skew 500 ppm
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:37.393746 21821 init.cc:377] Logged in from keytab as oryx/127.11.114.195@KRBTEST.COM (short username oryx)
I20260807 06:38:37.394130 21821 webserver.cc:460] Webserver started at http://127.11.114.195:38199/ using document root <none> and password file <none>
I20260807 06:38:37.394362 21821 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:37.394443 21821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:37.396174 21821 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:37.396781 21842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.396974 21821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:37.397085 21821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "0ecbfc604f3d4bc29742140f57361c18"
format_stamp: "Formatted at 2026-08-07 06:38:36 on dist-test-slave-gkw7"
server_key: "02e3197be2adcc17754b7aea3880d655"
server_key_iv: "e611dac009edb6e6e5b2986eec4462cd"
server_key_version: "encryptionkey@0"
I20260807 06:38:37.397413 21821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:37.416682 21821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:37.418962 21821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:37.419106 21821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:37.419390 21821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:37.419740 21821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:37.419797 21821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.419840 21821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:37.419869 21821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:37.426676 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.424026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.11.114.252:33895)
0807 06:38:37.424080 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.424083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.424095 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:38:37.424133 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.424134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.424164 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.424270 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:37.424416 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.424936 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.425904 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.426329 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.426425 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.426500 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.426565 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.426603 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:37.426681 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.423937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.0.0.1:45426)
0807 06:38:37.424006 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.424019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.424069 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.424278 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.424279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.424282 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:37.424346 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.424348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.425029 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.425031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.425804 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.425806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.426425 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.426426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.426492 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.426496 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.426558 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.426614 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 06:38:37.427054 21384 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 06:38:37.427273 21427 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27284d9f9c8e4ff4a0c93a466cc3af4d. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:37.427404 21821 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:38629
I20260807 06:38:37.427416 21427 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:38:37.427696 21999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:38629 every 8 connection(s)
I20260807 06:38:37.427909 21821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:38:37.431186 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 21821
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.252@KRBTEST.COM
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.253@KRBTEST.COM
Aug 07 06:38:37 dist-test-slave-gkw7 krb5kdc[20156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084717, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:37.441956 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.428672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:53503)
0807 06:38:37.428876 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.428921 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.429984 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.430223 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.430227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.430621 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.430819 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.430825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.431796 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.431797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.432978 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.432981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.433564 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.433565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.433640 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.434166 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.434177 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.437431 (+ 3254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.440091 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.440099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.440102 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.440436 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.440669 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.440669 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.440670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.440815 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.441114 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.441121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.441292 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.441426 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.441659 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":96,"thread_start_us":93,"threads_started":1}
I20260807 06:38:37.442402 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.428651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:53503 (local address 127.11.114.254:33755)
0807 06:38:37.428700 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.428702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.428994 (+ 292us) server_negotiation.cc:407] Connection header received
0807 06:38:37.430077 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.430078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.430090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.430122 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.430927 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.431660 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.433065 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.433342 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.433404 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.437566 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.437569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.437569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.437576 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.439996 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.440530 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.440532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.440532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.440595 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.440891 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.440893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.440893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.441022 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.441086 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.442220 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.442297 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:38:37.442768 22001 heartbeater.cc:347] Connected to a master server at 127.11.114.254:33755
I20260807 06:38:37.442873 22001 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.443090 22001 heartbeater.cc:511] Master 127.11.114.254:33755 requested a full tablet report, sending...
I20260807 06:38:37.443462 20217 ts_manager.cc:194] Re-registered known tserver with Master: 0ecbfc604f3d4bc29742140f57361c18 (127.11.114.195:38629)
I20260807 06:38:37.444015 20217 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:53503
I20260807 06:38:37.444361 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.433468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.11.114.253:39293)
0807 06:38:37.433509 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.433511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.434657 (+ 1146us) server_negotiation.cc:407] Connection header received
0807 06:38:37.435533 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.435534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.435545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.435606 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.436295 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.436998 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.438011 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.438337 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.438378 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.439978 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.439980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.439980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.439987 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.442193 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.443452 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.443453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.443454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.443514 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.443739 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.443740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.443740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.443871 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.443946 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.444182 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.444265 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:38:37.444447 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.429177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:58669 (local address 127.11.114.252:33895)
0807 06:38:37.429219 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.429221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.429239 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:38:37.430077 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.430078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.430090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.430122 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.430927 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.431665 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.433086 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.433390 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.433438 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.440870 (+ 7432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.440872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.440872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.440880 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.443195 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.443630 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.443631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.443632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.443690 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.443886 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.443886 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.443887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.443999 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.444040 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.444262 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.444343 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:38:37.445006 22006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.428985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.11.114.195:58669)
0807 06:38:37.429098 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.429107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.429987 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.430224 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.430238 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.430617 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.430819 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.430825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.431796 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.431797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.432978 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.432981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.433521 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.433522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.433616 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.434312 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.434319 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.440800 (+ 6481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.443339 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.443340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.443340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.443541 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.443765 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.443765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.443766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.443823 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.444089 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.444090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.444131 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.444556 (+ 425us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.444665 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":94,"thread_start_us":70,"threads_started":1}
I20260807 06:38:37.445369 22000 heartbeater.cc:347] Connected to a master server at 127.11.114.252:33895
I20260807 06:38:37.445415 22000 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.445446 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.433575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.11.114.254:33755)
0807 06:38:37.434232 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.434244 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.434428 (+ 184us) server_negotiation.cc:407] Connection header received
0807 06:38:37.434460 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.434461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.434472 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.434536 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.435457 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.436369 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.437561 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.437871 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.438176 (+ 305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.439529 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.439530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.439531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.439538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.443507 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.443942 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.443942 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.443943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.444015 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.444287 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.444287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.444288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.444795 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.445219 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.445260 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.445341 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":87,"threads_started":1}
I20260807 06:38:37.445822 22000 heartbeater.cc:511] Master 127.11.114.252:33895 requested a full tablet report, sending...
I20260807 06:38:37.446257 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.428926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:56493 (local address 127.11.114.253:39293)
0807 06:38:37.428976 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.428978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.428987 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:37.430076 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.430077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.430092 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.430151 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.430910 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.431690 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.432904 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.433193 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.433255 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.441189 (+ 7934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.441190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.441191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.441198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.442955 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.445370 (+ 2415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.445371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.445371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.445432 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.445687 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.445688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.445689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.445806 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.445849 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.446065 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.446139 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:37.446595 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.428672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.11.114.195:56493)
0807 06:38:37.428877 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.428922 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.429964 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.430223 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.430227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.430625 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:37.430818 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.430822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.431919 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.431920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.432807 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.432810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.433324 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.433325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.433489 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.434465 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:37.434470 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:37.441099 (+ 6629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:37.443042 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.443043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.443043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.443257 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.445512 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:37.445513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:37.445514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:37.445610 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:37.445899 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:37.445900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:37.445940 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.446335 (+ 395us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.446455 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":39,"spinlock_wait_cycles":1280,"thread_start_us":119,"threads_started":1}
I20260807 06:38:37.446401 21368 ts_manager.cc:194] Registered new tserver with Master: 0ecbfc604f3d4bc29742140f57361c18 (127.11.114.195:38629)
I20260807 06:38:37.446913 22002 heartbeater.cc:347] Connected to a master server at 127.11.114.253:39293
I20260807 06:38:37.446954 22002 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:37.446996 22002 heartbeater.cc:511] Master 127.11.114.253:39293 requested a full tablet report, sending...
I20260807 06:38:37.447248 21108 ts_manager.cc:194] Registered new tserver with Master: 0ecbfc604f3d4bc29742140f57361c18 (127.11.114.195:38629)
I20260807 06:38:37.448541 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.433840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.11.114.252:33895)
0807 06:38:37.435569 (+ 1729us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.435572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.435579 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:38:37.435598 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.435598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.435609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.435667 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.436400 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.437171 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.438245 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.438567 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.438635 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.440144 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.440146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.440146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.440153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.447340 (+ 7187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.447696 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.447697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.447697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.447759 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.448043 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.448044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.448044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.448204 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.448303 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.448423 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.448484 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1709,"thread_start_us":62,"threads_started":1}
I20260807 06:38:37.447336 20217 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:37.449508 20217 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:37.450263 21312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: NON_VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: true } } }
I20260807 06:38:37.450358 21312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:37.451617 22024 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: attempting to promote NON_VOTER ee5a25e82eca47e69d975cccd2148e37 to VOTER
I20260807 06:38:37.451754 22024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } }
I20260807 06:38:37.452152 21121 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Refusing update from remote peer 27284d9f9c8e4ff4a0c93a466cc3af4d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 06:38:37.452157 21384 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 LEARNER]: Refusing update from remote peer 27284d9f9c8e4ff4a0c93a466cc3af4d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 06:38:37.452353 22022 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 06:38:37.452458 21312 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 06:38:37.453085 21312 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:37.453249 21121 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:37.453665 21384 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:37.454078 21312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:37.454149 21312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:37.454202 21312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:37.454242 21312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:37.454317 21312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:37.454366 21312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:37.454334 22037 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:37.454421 22037 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:38:37.454839 21427 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:37.454932 21427 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:38:37.459709 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.455837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38629 (local address 127.0.0.1:60932)
0807 06:38:37.455972 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.455984 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.456027 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.456360 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.456361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.456364 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:37.456418 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.456419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.457526 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.457527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.458512 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.458514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.459464 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.459465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.459526 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.459530 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.459595 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.459630 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":95,"threads_started":1}
I20260807 06:38:37.459738 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.455997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60932 (local address 127.11.114.195:38629)
0807 06:38:37.456158 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.456163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.456173 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:37.456212 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.456214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.456241 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.456493 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:37.456545 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.457435 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.458611 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.459374 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.459474 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.459540 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.459611 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.459655 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260807 06:38:37.459921 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.455786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:37155 (local address 127.0.0.1:39740)
0807 06:38:37.455972 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.455984 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.456035 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.456483 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.456484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.456486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:37.456533 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.456534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.457853 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.457854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.458688 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.458690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.459532 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.459533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.459601 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.459605 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.459637 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.459673 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":1}
I20260807 06:38:37.460280 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.455861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44647 (local address 127.0.0.1:57040)
0807 06:38:37.456358 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.456369 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.456412 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.456719 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.456720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.456722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:37.456778 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.456779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.458031 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.458032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.458953 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.458955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.459988 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.459989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.460056 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.460060 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.460140 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.460194 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":91,"threads_started":1}
I20260807 06:38:37.460304 22045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.456182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.11.114.193:44647)
0807 06:38:37.456524 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.456528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.456538 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:37.456575 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.456577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.456593 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.456662 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:37.456846 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.457949 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.459048 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.459799 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.459893 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.459951 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.460158 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.460210 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":84,"threads_started":1}
I20260807 06:38:37.460511 21908 tablet_service.cc:1467] Processing CreateTablet for tablet 90cc90e8b25f4f8c8840369f13bda12c (DEFAULT_TABLE table=test-table [id=de77da3f10634868af7c4ab51ddb527f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:37.460793 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.456125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.11.114.194:37155)
0807 06:38:37.456306 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.456311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.456322 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:38:37.456363 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.456365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.456379 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.456486 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:37.456589 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.457756 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.458790 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.459438 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.460142 (+ 704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.460232 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.460258 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.460301 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
I20260807 06:38:37.460922 21908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90cc90e8b25f4f8c8840369f13bda12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:37.460875 21534 tablet_service.cc:1467] Processing CreateTablet for tablet 90cc90e8b25f4f8c8840369f13bda12c (DEFAULT_TABLE table=test-table [id=de77da3f10634868af7c4ab51ddb527f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:37.461122 21534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90cc90e8b25f4f8c8840369f13bda12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:37.461558 21728 tablet_service.cc:1467] Processing CreateTablet for tablet 90cc90e8b25f4f8c8840369f13bda12c (DEFAULT_TABLE table=test-table [id=de77da3f10634868af7c4ab51ddb527f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:37.461920 21728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90cc90e8b25f4f8c8840369f13bda12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:37.464540 22046 tablet_bootstrap.cc:492] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Bootstrap starting.
I20260807 06:38:37.464779 22047 tablet_bootstrap.cc:492] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: Bootstrap starting.
I20260807 06:38:37.465335 22048 tablet_bootstrap.cc:492] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Bootstrap starting.
I20260807 06:38:37.465636 22046 tablet_bootstrap.cc:654] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:37.465451 22047 tablet_bootstrap.cc:654] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:37.466009 22046 log.cc:824] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:37.466101 22048 tablet_bootstrap.cc:654] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:37.466393 22048 log.cc:824] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:37.467321 22048 tablet_bootstrap.cc:492] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: No bootstrap required, opened a new log
I20260807 06:38:37.467403 22048 ts_tablet_manager.cc:1397] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:37.468080 22047 log.cc:824] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:37.468940 22047 tablet_bootstrap.cc:492] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: No bootstrap required, opened a new log
I20260807 06:38:37.469007 22047 ts_tablet_manager.cc:1397] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:38:37.470574 22048 raft_consensus.cc:348] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.470748 22048 raft_consensus.cc:374] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:37.470786 22048 raft_consensus.cc:729] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ddfc1398ad41d98c620c704c2e2ad4, State: Initialized, Role: FOLLOWER
I20260807 06:38:37.470906 22048 consensus_queue.cc:260] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.471246 22048 ts_tablet_manager.cc:1428] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 06:38:37.471403 21814 heartbeater.cc:503] Master 127.11.114.254:33755 was elected leader, sending a full tablet report...
I20260807 06:38:37.471581 22047 raft_consensus.cc:348] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.471745 22047 raft_consensus.cc:374] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:37.471786 22047 raft_consensus.cc:729] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecbfc604f3d4bc29742140f57361c18, State: Initialized, Role: FOLLOWER
I20260807 06:38:37.471905 22047 consensus_queue.cc:260] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.472159 22047 ts_tablet_manager.cc:1428] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:38:37.472286 22001 heartbeater.cc:503] Master 127.11.114.254:33755 was elected leader, sending a full tablet report...
I20260807 06:38:37.472667 22046 tablet_bootstrap.cc:492] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: No bootstrap required, opened a new log
I20260807 06:38:37.472728 22046 ts_tablet_manager.cc:1397] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260807 06:38:37.475276 22046 raft_consensus.cc:348] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.475390 22046 raft_consensus.cc:374] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:37.475410 22046 raft_consensus.cc:729] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110dec602478421ba7f0fd9d6806ab03, State: Initialized, Role: FOLLOWER
I20260807 06:38:37.475531 22046 consensus_queue.cc:260] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.475780 22046 ts_tablet_manager.cc:1428] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 06:38:37.475848 21628 heartbeater.cc:503] Master 127.11.114.254:33755 was elected leader, sending a full tablet report...
W20260807 06:38:37.524156 21817 tablet.cc:2369] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:37.587495 22054 raft_consensus.cc:482] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:37.587627 22054 raft_consensus.cc:504] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.588011 22054 leader_election.cc:288] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155), 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647)
I20260807 06:38:37.591074 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.588251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44647 (local address 127.11.114.195:43143)
0807 06:38:37.588380 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.588391 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.588439 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.588647 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.588648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.588649 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:37.588725 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.588727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.589364 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.589365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.590199 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.590202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.590829 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.590830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.590905 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.590907 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.590949 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.590980 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":50}
I20260807 06:38:37.591087 22045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.588463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:43143 (local address 127.11.114.193:44647)
0807 06:38:37.588533 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.588535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.588544 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:37.588568 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.588569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.588578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.588637 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:37.588803 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.589259 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.590302 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.590752 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.590810 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.590840 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.590972 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.591020 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:38:37.591168 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.588297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:41665 (local address 127.11.114.194:37155)
0807 06:38:37.588365 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.588367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.588380 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:38:37.588436 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.588437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.588448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.588531 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:37.588722 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.589232 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.590335 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.590765 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.590856 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.590865 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.591041 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.591077 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:38:37.591205 22006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.588216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:37155 (local address 127.11.114.195:41665)
0807 06:38:37.588325 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:37.588336 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:37.588378 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:37.588549 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:37.588550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:37.588553 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:37.588638 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.588640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.589326 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.589327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.590215 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:37.590218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.590871 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.590872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:37.590964 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.590967 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:37.591024 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:38:37.591064 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":52}
I20260807 06:38:37.591521 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90cc90e8b25f4f8c8840369f13bda12c" candidate_uuid: "0ecbfc604f3d4bc29742140f57361c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110dec602478421ba7f0fd9d6806ab03" is_pre_election: true
I20260807 06:38:37.591517 21757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90cc90e8b25f4f8c8840369f13bda12c" candidate_uuid: "0ecbfc604f3d4bc29742140f57361c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" is_pre_election: true
I20260807 06:38:37.591708 21757 raft_consensus.cc:2454] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecbfc604f3d4bc29742140f57361c18 in term 0.
I20260807 06:38:37.591715 21563 raft_consensus.cc:2454] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecbfc604f3d4bc29742140f57361c18 in term 0.
I20260807 06:38:37.591964 21845 leader_election.cc:302] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecbfc604f3d4bc29742140f57361c18, 31ddfc1398ad41d98c620c704c2e2ad4; no voters:
I20260807 06:38:37.592113 22054 raft_consensus.cc:2781] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:37.592170 22054 raft_consensus.cc:482] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:37.592206 22054 raft_consensus.cc:3037] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:37.592983 22054 raft_consensus.cc:504] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.593134 22054 leader_election.cc:288] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [CANDIDATE]: Term 1 election: Requested vote from peers 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155), 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647)
I20260807 06:38:37.593278 21757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90cc90e8b25f4f8c8840369f13bda12c" candidate_uuid: "0ecbfc604f3d4bc29742140f57361c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ddfc1398ad41d98c620c704c2e2ad4"
I20260807 06:38:37.593310 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90cc90e8b25f4f8c8840369f13bda12c" candidate_uuid: "0ecbfc604f3d4bc29742140f57361c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110dec602478421ba7f0fd9d6806ab03"
I20260807 06:38:37.593364 21757 raft_consensus.cc:3037] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:37.593375 21563 raft_consensus.cc:3037] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:37.594297 21757 raft_consensus.cc:2454] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecbfc604f3d4bc29742140f57361c18 in term 1.
I20260807 06:38:37.594297 21563 raft_consensus.cc:2454] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecbfc604f3d4bc29742140f57361c18 in term 1.
I20260807 06:38:37.594522 21850 leader_election.cc:302] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecbfc604f3d4bc29742140f57361c18, 110dec602478421ba7f0fd9d6806ab03; no voters:
I20260807 06:38:37.594656 22054 raft_consensus.cc:2781] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:37.594836 22054 raft_consensus.cc:686] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 1 LEADER]: Becoming Leader. State: Replica: 0ecbfc604f3d4bc29742140f57361c18, State: Running, Role: LEADER
I20260807 06:38:37.594947 22054 consensus_queue.cc:237] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } }
I20260807 06:38:37.595974 20217 catalog_manager.cc:5495] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ecbfc604f3d4bc29742140f57361c18 (127.11.114.195). New cstate: current_term: 1 leader_uuid: "0ecbfc604f3d4bc29742140f57361c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } health_report { overall_health: HEALTHY } } }
I20260807 06:38:37.601629 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.599398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.11.114.195:38629)
0807 06:38:37.599436 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.599437 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.599446 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:37.599513 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.599514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.599529 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.599590 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:37.599675 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.600195 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.600881 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.601059 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.601109 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.601309 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:37.601397 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:37.601459 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.601491 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.601508 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 06:38:37.602862 21630 tablet.cc:2369] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:37.603567 21757 raft_consensus.cc:1261] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 1 FOLLOWER]: Refusing update from remote peer 0ecbfc604f3d4bc29742140f57361c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:37.603626 21563 raft_consensus.cc:1261] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 1 FOLLOWER]: Refusing update from remote peer 0ecbfc604f3d4bc29742140f57361c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:37.603919 22054 consensus_queue.cc:1035] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:37.604049 22057 consensus_queue.cc:1035] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:37.607908 22062 mvcc.cc:205] Tried to move back new op lower bound from 7315803003301507072 to 7315803003269627904. Current Snapshot: MvccSnapshot[applied={T|T < 7315803003301507072}]
I20260807 06:38:37.609159 20214 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:37.609256 20214 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:37.610715 20214 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27284d9f9c8e4ff4a0c93a466cc3af4d: Sending DeleteTablet for 3 replicas of tablet 90cc90e8b25f4f8c8840369f13bda12c
I20260807 06:38:37.611032 21728 tablet_service.cc:1514] Processing DeleteTablet for tablet 90cc90e8b25f4f8c8840369f13bda12c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:37 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:39740
I20260807 06:38:37.611055 21908 tablet_service.cc:1514] Processing DeleteTablet for tablet 90cc90e8b25f4f8c8840369f13bda12c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:37 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:60932
I20260807 06:38:37.611307 21534 tablet_service.cc:1514] Processing DeleteTablet for tablet 90cc90e8b25f4f8c8840369f13bda12c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:37 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:57040
I20260807 06:38:37.611554 22077 tablet_replica.cc:330] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: stopping tablet replica
I20260807 06:38:37.611657 22077 raft_consensus.cc:2229] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:37.611753 22077 raft_consensus.cc:2258] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:37.612174 22077 ts_tablet_manager.cc:1905] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:37.612859 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 20175
I20260807 06:38:37.613931 22077 ts_tablet_manager.cc:1918] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:37.614006 22077 log.cc:1196] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/90cc90e8b25f4f8c8840369f13bda12c
I20260807 06:38:37.614282 22077 ts_tablet_manager.cc:1939] T 90cc90e8b25f4f8c8840369f13bda12c P 110dec602478421ba7f0fd9d6806ab03: Deleting consensus metadata
I20260807 06:38:37.621680 22075 tablet_replica.cc:330] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: stopping tablet replica
I20260807 06:38:37.621680 22076 tablet_replica.cc:330] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: stopping tablet replica
I20260807 06:38:37.621834 22076 raft_consensus.cc:2229] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:37.621834 22075 raft_consensus.cc:2229] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:37.621956 22075 raft_consensus.cc:2258] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:37.621965 22076 raft_consensus.cc:2258] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:37.623091 22076 ts_tablet_manager.cc:1905] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:37.624004 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.611692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.11.114.252:33895)
0807 06:38:37.611734 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.611736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.611744 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:37.611764 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.611764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.611776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.611825 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.612577 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.613434 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.614804 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.615132 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.615213 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.617119 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.617121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.617121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.617146 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.622450 (+ 5304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.622902 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.622904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.622904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.622965 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.623285 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.623285 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.623286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.623443 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.623526 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.623748 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.623855 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:38:37.625000 22076 ts_tablet_manager.cc:1918] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:37.625092 22076 log.cc:1196] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/90cc90e8b25f4f8c8840369f13bda12c
I20260807 06:38:37.625375 22076 ts_tablet_manager.cc:1939] T 90cc90e8b25f4f8c8840369f13bda12c P 0ecbfc604f3d4bc29742140f57361c18: Deleting consensus metadata
I20260807 06:38:37.626271 22075 ts_tablet_manager.cc:1905] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:37.627709 22075 ts_tablet_manager.cc:1918] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:37.627815 22075 log.cc:1196] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/90cc90e8b25f4f8c8840369f13bda12c
I20260807 06:38:37.628070 22075 ts_tablet_manager.cc:1939] T 90cc90e8b25f4f8c8840369f13bda12c P 31ddfc1398ad41d98c620c704c2e2ad4: Deleting consensus metadata
W20260807 06:38:37.630532 21464 connection.cc:535] server connection from 127.0.0.1:57040 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 06:38:37.630555 21848 connection.cc:535] server connection from 127.0.0.1:60932 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 06:38:37.630712 21665 connection.cc:535] server connection from 127.0.0.1:39740 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:38:37.640513 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.632854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55446 (local address 127.11.114.253:39293)
0807 06:38:37.632902 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.632904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.632913 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:37.632940 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.632941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.632953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.633013 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.633853 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.634651 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.635734 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.635982 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.636060 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.637316 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.637317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.637318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.637325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.639135 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.639557 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.639558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.639559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.639618 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.639845 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.639846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.639846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.639971 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.640067 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.640310 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.640388 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:38:37.640729 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:37.632760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.11.114.252:33895)
0807 06:38:37.632803 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:37.632805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:37.632937 (+ 132us) server_negotiation.cc:407] Connection header received
0807 06:38:37.632987 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:37.632988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:37.632999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:37.633048 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:37.634096 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.634805 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.635500 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:37.635762 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:37.635827 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:37.637613 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:37.637615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:37.637615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:37.637623 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:37.639268 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.639744 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.639745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.639746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.639800 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:37.640022 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:37.640022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:37.640023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:37.640154 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:37.640250 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:37.640524 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 06:38:37.640612 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:38:38.609941 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:38.609561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:52189)
0807 06:38:38.609824 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:38.609897 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":88,"threads_started":1}
I20260807 06:38:38.609951 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:38.609722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:45533)
0807 06:38:38.609841 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:38.609902 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":41,"threads_started":1}
I20260807 06:38:38.609951 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:38.609698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:58021)
0807 06:38:38.609826 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:38.609903 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
W20260807 06:38:38.610167 21814 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:33755 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
W20260807 06:38:38.610220 21628 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:33755 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
W20260807 06:38:38.610337 22001 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:33755 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:39.111402 22090 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27284d9f9c8e4ff4a0c93a466cc3af4d)
I20260807 06:38:39.111538 22090 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } }
I20260807 06:38:39.111837 22090 leader_election.cc:288] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755), bbd1d4d07d1e4bd885041290cffa1e1b (127.11.114.253:39293)
I20260807 06:38:39.111979 22092 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27284d9f9c8e4ff4a0c93a466cc3af4d)
I20260807 06:38:39.112129 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.111935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49326)
0807 06:38:39.112039 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.112079 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":40,"threads_started":1}
I20260807 06:38:39.112112 22092 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } }
W20260807 06:38:39.112282 21348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:39.112540 22092 leader_election.cc:288] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755), ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252:33895)
I20260807 06:38:39.112709 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.112446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49334)
0807 06:38:39.112595 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.112666 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
W20260807 06:38:39.112838 21082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:39.113003 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.112923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49340)
0807 06:38:39.112958 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.112980 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 06:38:39.113080 21348 leader_election.cc:334] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755): Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:39.113355 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.113247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49354)
0807 06:38:39.113303 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.113336 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 06:38:39.113510 21082 leader_election.cc:334] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755): Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:39.116113 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.112596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.11.114.252:33895)
0807 06:38:39.112738 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:39.112740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:39.112747 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:38:39.112768 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:39.112769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:39.112779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:39.112839 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:39.113169 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:39.113883 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:39.115241 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:39.115779 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:39.115844 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:39.115858 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:39.116000 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 06:38:39.116045 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260807 06:38:39.116117 22095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.112149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.11.114.253:39293)
0807 06:38:39.112410 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:39.112412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:39.112420 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:39.112449 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:39.112451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:39.112467 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:39.112820 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:39.112863 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:39.113669 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:39.115125 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:39.115691 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:39.115772 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:39.115782 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:39.115959 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:38:39.116033 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":237,"thread_start_us":40,"threads_started":1}
I20260807 06:38:39.116119 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.111984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:39293 (local address 127.0.0.1:55454)
0807 06:38:39.112113 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.112124 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:39.112300 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:39.112560 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:39.112561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:39.112563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:39.112633 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:39.112636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:39.113766 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:39.113767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:39.114993 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:39.114996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:39.115795 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:39.115796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:39.115883 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:39.115888 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:39.115946 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 06:38:39.115999 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260807 06:38:39.116118 22097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.112464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:33895 (local address 127.0.0.1:45456)
0807 06:38:39.112658 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.112668 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:39.112723 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:39.112906 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:39.112906 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:39.112909 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:39.112967 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:39.112970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:39.113967 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:39.113968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:39.115157 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:39.115160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:39.115860 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:39.115861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:39.115926 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:39.115931 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:39.115984 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 06:38:39.116030 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":45,"threads_started":1}
I20260807 06:38:39.116490 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee5a25e82eca47e69d975cccd2148e37" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" is_pre_election: true
I20260807 06:38:39.116607 21121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5a25e82eca47e69d975cccd2148e37 in term 1.
I20260807 06:38:39.116627 21384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ee5a25e82eca47e69d975cccd2148e37" is_pre_election: true
I20260807 06:38:39.116732 21384 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd1d4d07d1e4bd885041290cffa1e1b in term 1.
I20260807 06:38:39.116788 21352 leader_election.cc:302] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbd1d4d07d1e4bd885041290cffa1e1b, ee5a25e82eca47e69d975cccd2148e37; no voters: 27284d9f9c8e4ff4a0c93a466cc3af4d
I20260807 06:38:39.116861 21083 leader_election.cc:302] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbd1d4d07d1e4bd885041290cffa1e1b, ee5a25e82eca47e69d975cccd2148e37; no voters: 27284d9f9c8e4ff4a0c93a466cc3af4d
I20260807 06:38:39.116888 22090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 06:38:39.116945 22090 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 27284d9f9c8e4ff4a0c93a466cc3af4d)
I20260807 06:38:39.116968 22090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:38:39.116961 22092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 06:38:39.116995 22092 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 27284d9f9c8e4ff4a0c93a466cc3af4d)
I20260807 06:38:39.117014 22092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:38:39.117893 22092 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } }
I20260807 06:38:39.117893 22090 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } }
I20260807 06:38:39.118037 22090 leader_election.cc:288] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 2 election: Requested vote from peers 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755), bbd1d4d07d1e4bd885041290cffa1e1b (127.11.114.253:39293)
I20260807 06:38:39.118202 22092 leader_election.cc:288] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [CANDIDATE]: Term 2 election: Requested vote from peers 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755), ee5a25e82eca47e69d975cccd2148e37 (127.11.114.252:33895)
I20260807 06:38:39.118216 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee5a25e82eca47e69d975cccd2148e37" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b"
I20260807 06:38:39.118319 21121 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5a25e82eca47e69d975cccd2148e37 in current term 2: Already voted for candidate bbd1d4d07d1e4bd885041290cffa1e1b in this term.
I20260807 06:38:39.118337 21384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ee5a25e82eca47e69d975cccd2148e37"
I20260807 06:38:39.118422 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.118166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49358)
0807 06:38:39.118355 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.118391 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260807 06:38:39.118449 21384 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bbd1d4d07d1e4bd885041290cffa1e1b in current term 2: Already voted for candidate ee5a25e82eca47e69d975cccd2148e37 in this term.
I20260807 06:38:39.118630 22097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.118159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49356)
0807 06:38:39.118576 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.118607 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402}
I20260807 06:38:39.118769 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.118659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49374)
0807 06:38:39.118716 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.118745 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 06:38:39.118834 21348 leader_election.cc:334] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755): Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:39.118871 21348 leader_election.cc:302] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5a25e82eca47e69d975cccd2148e37; no voters: 27284d9f9c8e4ff4a0c93a466cc3af4d, bbd1d4d07d1e4bd885041290cffa1e1b
I20260807 06:38:39.118974 22090 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 06:38:39.118989 22097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.118879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49378)
0807 06:38:39.118933 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.118963 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 06:38:39.119092 21082 leader_election.cc:334] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755): Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:39.119165 21082 leader_election.cc:302] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bbd1d4d07d1e4bd885041290cffa1e1b; no voters: 27284d9f9c8e4ff4a0c93a466cc3af4d, ee5a25e82eca47e69d975cccd2148e37
I20260807 06:38:39.119256 22092 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 06:38:39.610898 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.610670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:59609)
0807 06:38:39.610804 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.610848 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260807 06:38:39.611028 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.610806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:39433)
0807 06:38:39.610937 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.610978 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260807 06:38:39.611032 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:39.610490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:53595)
0807 06:38:39.610958 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:39.610994 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":260,"threads_started":1}
I20260807 06:38:40.611655 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.611432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:52299)
0807 06:38:40.611564 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.611602 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260807 06:38:40.611655 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.611434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:58729)
0807 06:38:40.611564 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.611602 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260807 06:38:40.611682 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.611490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:52735)
0807 06:38:40.611600 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.611643 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":65,"threads_started":1}
I20260807 06:38:40.818928 22108 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:40.819027 22108 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } }
I20260807 06:38:40.819288 22108 leader_election.cc:288] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755), bbd1d4d07d1e4bd885041290cffa1e1b (127.11.114.253:39293)
I20260807 06:38:40.819581 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee5a25e82eca47e69d975cccd2148e37" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" is_pre_election: true
I20260807 06:38:40.819666 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.819432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49636)
0807 06:38:40.819584 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.819623 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260807 06:38:40.819702 21121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5a25e82eca47e69d975cccd2148e37 in term 2.
I20260807 06:38:40.819878 21352 leader_election.cc:302] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd1d4d07d1e4bd885041290cffa1e1b, ee5a25e82eca47e69d975cccd2148e37; no voters:
I20260807 06:38:40.819964 22108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 06:38:40.820003 22108 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:40.820029 22108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:38:40.820138 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.820031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49648)
0807 06:38:40.820085 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.820111 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 06:38:40.820230 21348 leader_election.cc:334] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755): Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:40.820847 22108 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } }
I20260807 06:38:40.820991 22108 leader_election.cc:288] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 3 election: Requested vote from peers 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755), bbd1d4d07d1e4bd885041290cffa1e1b (127.11.114.253:39293)
I20260807 06:38:40.821200 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.821101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49656)
0807 06:38:40.821140 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.821175 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 06:38:40.821182 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee5a25e82eca47e69d975cccd2148e37" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b"
I20260807 06:38:40.821259 21121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:38:40.821648 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.821516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49658)
0807 06:38:40.821588 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.821623 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 06:38:40.821763 21348 leader_election.cc:334] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755): Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
I20260807 06:38:40.822089 21121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5a25e82eca47e69d975cccd2148e37 in term 3.
I20260807 06:38:40.822274 21352 leader_election.cc:302] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbd1d4d07d1e4bd885041290cffa1e1b, ee5a25e82eca47e69d975cccd2148e37; no voters: 27284d9f9c8e4ff4a0c93a466cc3af4d
I20260807 06:38:40.822356 22108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:38:40.822469 22108 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [term 3 LEADER]: Becoming Leader. State: Replica: ee5a25e82eca47e69d975cccd2148e37, State: Running, Role: LEADER
I20260807 06:38:40.822568 22108 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } }
I20260807 06:38:40.822937 22112 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee5a25e82eca47e69d975cccd2148e37. Latest consensus state: current_term: 3 leader_uuid: "ee5a25e82eca47e69d975cccd2148e37" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:40.823045 22112 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:40.823328 22117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:40.823930 22117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:40.824036 22117 catalog_manager.cc:1259] Loaded cluster ID: 87ae0a6e997546f9bdd84ff1a1d87d4c
I20260807 06:38:40.824075 22117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:40.824250 22117 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:40.824407 22117 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Loaded TSK: 0
I20260807 06:38:40.824586 22117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:38:40.885392 21367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:40.887199 21367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:40.888243 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.888117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49678)
0807 06:38:40.888176 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.888208 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 06:38:40.888266 21121 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [term 3 FOLLOWER]: Refusing update from remote peer ee5a25e82eca47e69d975cccd2148e37: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 06:38:40.888511 22112 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 06:38:40.888690 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.888566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.0.0.1:49680)
0807 06:38:40.888611 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.888658 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:38:40.888762 21348 consensus_peers.cc:487] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 -> Peer 27284d9f9c8e4ff4a0c93a466cc3af4d (127.11.114.254:33755): Couldn't send request to peer 27284d9f9c8e4ff4a0c93a466cc3af4d. Status: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 06:38:40.889241 22120 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee5a25e82eca47e69d975cccd2148e37. Latest consensus state: current_term: 3 leader_uuid: "ee5a25e82eca47e69d975cccd2148e37" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:40.889358 22120 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:38:40.889819 22108 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ee5a25e82eca47e69d975cccd2148e37" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:40.889917 22108 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:40.890046 22120 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ee5a25e82eca47e69d975cccd2148e37" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "27284d9f9c8e4ff4a0c93a466cc3af4d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33755 } } peers { permanent_uuid: "bbd1d4d07d1e4bd885041290cffa1e1b" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 39293 } attrs { promote: false } } peers { permanent_uuid: "ee5a25e82eca47e69d975cccd2148e37" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 33895 } attrs { promote: false } } }
I20260807 06:38:40.890128 22120 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd1d4d07d1e4bd885041290cffa1e1b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:38:40.890106 22121 mvcc.cc:205] Tried to move back new op lower bound from 7315803016756289536 to 7315803016489967616. Current Snapshot: MvccSnapshot[applied={T|T < 7315803016756289536}]
I20260807 06:38:40.894922 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.891554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:37155 (local address 127.0.0.1:39746)
0807 06:38:40.891691 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.891700 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:40.891739 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:40.892050 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:40.892051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:40.892053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:40.892103 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:40.892105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.893098 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.893099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:40.893928 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:40.893930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.894673 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.894674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:40.894739 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:40.894743 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:40.894809 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:38:40.894852 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260807 06:38:40.894933 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.891622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39746 (local address 127.11.114.194:37155)
0807 06:38:40.891898 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:40.891901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:40.891909 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:40.891937 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:40.891937 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:40.891948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:40.892171 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:40.892209 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.892973 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.894018 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.894580 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.894674 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:40.894683 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:40.894827 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 06:38:40.894867 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":44,"threads_started":1}
I20260807 06:38:40.895200 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.891499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38629 (local address 127.0.0.1:60940)
0807 06:38:40.891538 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.891549 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:40.891606 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:40.892007 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:40.892009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:40.892010 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:40.892075 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:40.892083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.892945 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.892946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:40.894189 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:40.894192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.894965 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.894966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:40.895030 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:40.895035 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:40.895070 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 06:38:40.895130 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":4}
I20260807 06:38:40.895277 21728 tablet_service.cc:1467] Processing CreateTablet for tablet 95b32419135e4fba9540b9806f2e22b2 (DEFAULT_TABLE table=test-table [id=8023e23df1ec47458007f4ca5e3c06e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:40.895411 21728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b32419135e4fba9540b9806f2e22b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:40.895511 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.891739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60940 (local address 127.11.114.195:38629)
0807 06:38:40.891862 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:40.891864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:40.891872 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:40.891898 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:40.891898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:40.891909 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:40.892006 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:40.892224 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.892860 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.894298 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.894868 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.895352 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:40.895365 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:40.895388 (+ 23us) server_negotiation.cc:299] Negotiation successful
0807 06:38:40.895439 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260807 06:38:40.895777 21908 tablet_service.cc:1467] Processing CreateTablet for tablet 95b32419135e4fba9540b9806f2e22b2 (DEFAULT_TABLE table=test-table [id=8023e23df1ec47458007f4ca5e3c06e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:40.895925 21908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b32419135e4fba9540b9806f2e22b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:40.896620 22135 tablet_bootstrap.cc:492] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: Bootstrap starting.
I20260807 06:38:40.896901 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.891591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44647 (local address 127.0.0.1:57054)
0807 06:38:40.891726 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.891747 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:40.891800 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:40.892067 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:40.892068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:40.892070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:40.892104 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:40.892105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.893080 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.893081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:40.894335 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:40.894337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.895127 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.895129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:40.896712 (+ 1583us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:40.896716 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:40.896790 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:38:40.896835 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260807 06:38:40.897027 22135 tablet_bootstrap.cc:654] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:40.897162 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.891780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57054 (local address 127.11.114.193:44647)
0807 06:38:40.891898 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:40.891901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:40.891909 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:40.891937 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:40.891938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:40.891948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:40.892020 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:40.892178 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.892971 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.894455 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:40.895056 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:40.895128 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:40.895137 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:40.897003 (+ 1866us) server_negotiation.cc:299] Negotiation successful
0807 06:38:40.897059 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
I20260807 06:38:40.897372 22137 tablet_bootstrap.cc:492] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: Bootstrap starting.
I20260807 06:38:40.897404 21534 tablet_service.cc:1467] Processing CreateTablet for tablet 95b32419135e4fba9540b9806f2e22b2 (DEFAULT_TABLE table=test-table [id=8023e23df1ec47458007f4ca5e3c06e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:40.897552 21534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b32419135e4fba9540b9806f2e22b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:40.897790 22137 tablet_bootstrap.cc:654] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:40.898339 22137 tablet_bootstrap.cc:492] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: No bootstrap required, opened a new log
I20260807 06:38:40.898372 22137 ts_tablet_manager.cc:1397] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:40.898468 22137 raft_consensus.cc:348] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:40.898511 22137 raft_consensus.cc:374] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:40.898523 22137 raft_consensus.cc:729] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecbfc604f3d4bc29742140f57361c18, State: Initialized, Role: FOLLOWER
I20260807 06:38:40.898586 22137 consensus_queue.cc:260] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:40.898752 22140 tablet_bootstrap.cc:492] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03: Bootstrap starting.
I20260807 06:38:40.898770 22137 ts_tablet_manager.cc:1428] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:40.899080 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.898984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:36395)
0807 06:38:40.899029 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.899055 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 06:38:40.899178 22140 tablet_bootstrap.cc:654] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:40.899396 22000 heartbeater.cc:503] Master 127.11.114.252:33895 was elected leader, sending a full tablet report...
I20260807 06:38:40.899770 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.899667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:38377)
0807 06:38:40.899715 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.899743 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 06:38:40.900360 22135 tablet_bootstrap.cc:492] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: No bootstrap required, opened a new log
I20260807 06:38:40.900427 22135 ts_tablet_manager.cc:1397] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:38:40.900532 22140 tablet_bootstrap.cc:492] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03: No bootstrap required, opened a new log
I20260807 06:38:40.900600 22140 ts_tablet_manager.cc:1397] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:40.900552 22135 raft_consensus.cc:348] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:40.900630 22135 raft_consensus.cc:374] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:40.900661 22135 raft_consensus.cc:729] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ddfc1398ad41d98c620c704c2e2ad4, State: Initialized, Role: FOLLOWER
I20260807 06:38:40.900710 22135 consensus_queue.cc:260] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:40.900731 22140 raft_consensus.cc:348] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:40.900835 22140 raft_consensus.cc:374] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:40.900871 22140 raft_consensus.cc:729] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110dec602478421ba7f0fd9d6806ab03, State: Initialized, Role: FOLLOWER
I20260807 06:38:40.900877 22135 ts_tablet_manager.cc:1428] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:40.900923 22140 consensus_queue.cc:260] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:40.901171 22140 ts_tablet_manager.cc:1428] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:40.901198 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.901110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:34683)
0807 06:38:40.901147 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.901174 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 06:38:40.901561 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.901448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:56731)
0807 06:38:40.901483 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.901505 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 06:38:40.901652 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.901511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:41487)
0807 06:38:40.901594 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.901625 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 06:38:40.901865 21812 heartbeater.cc:503] Master 127.11.114.252:33895 was elected leader, sending a full tablet report...
I20260807 06:38:40.902309 21627 heartbeater.cc:503] Master 127.11.114.252:33895 was elected leader, sending a full tablet report...
I20260807 06:38:40.902529 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:40.902448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:49809)
0807 06:38:40.902480 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:40.902506 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260807 06:38:40.930847 22003 tablet.cc:2369] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:41.050493 22141 raft_consensus.cc:482] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:41.050602 22141 raft_consensus.cc:504] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:41.050817 22141 leader_election.cc:288] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155), 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647)
I20260807 06:38:41.051154 21757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b32419135e4fba9540b9806f2e22b2" candidate_uuid: "0ecbfc604f3d4bc29742140f57361c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" is_pre_election: true
I20260807 06:38:41.051270 21757 raft_consensus.cc:2454] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecbfc604f3d4bc29742140f57361c18 in term 0.
I20260807 06:38:41.051182 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b32419135e4fba9540b9806f2e22b2" candidate_uuid: "0ecbfc604f3d4bc29742140f57361c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110dec602478421ba7f0fd9d6806ab03" is_pre_election: true
I20260807 06:38:41.051333 21563 raft_consensus.cc:2454] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecbfc604f3d4bc29742140f57361c18 in term 0.
I20260807 06:38:41.051471 21845 leader_election.cc:302] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecbfc604f3d4bc29742140f57361c18, 31ddfc1398ad41d98c620c704c2e2ad4; no voters:
I20260807 06:38:41.051589 22141 raft_consensus.cc:2781] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:41.051635 22141 raft_consensus.cc:482] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:41.051651 22141 raft_consensus.cc:3037] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:41.052399 22141 raft_consensus.cc:504] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:41.052667 22141 leader_election.cc:288] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [CANDIDATE]: Term 1 election: Requested vote from peers 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155), 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647)
I20260807 06:38:41.052714 21757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b32419135e4fba9540b9806f2e22b2" candidate_uuid: "0ecbfc604f3d4bc29742140f57361c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ddfc1398ad41d98c620c704c2e2ad4"
I20260807 06:38:41.052722 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b32419135e4fba9540b9806f2e22b2" candidate_uuid: "0ecbfc604f3d4bc29742140f57361c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110dec602478421ba7f0fd9d6806ab03"
I20260807 06:38:41.052806 21563 raft_consensus.cc:3037] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:41.052806 21757 raft_consensus.cc:3037] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:41.053733 21757 raft_consensus.cc:2454] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecbfc604f3d4bc29742140f57361c18 in term 1.
I20260807 06:38:41.053733 21563 raft_consensus.cc:2454] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecbfc604f3d4bc29742140f57361c18 in term 1.
I20260807 06:38:41.053956 21850 leader_election.cc:302] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecbfc604f3d4bc29742140f57361c18, 110dec602478421ba7f0fd9d6806ab03; no voters:
I20260807 06:38:41.054121 22141 raft_consensus.cc:2781] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:41.054327 22141 raft_consensus.cc:686] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 1 LEADER]: Becoming Leader. State: Replica: 0ecbfc604f3d4bc29742140f57361c18, State: Running, Role: LEADER
I20260807 06:38:41.054378 22141 consensus_queue.cc:237] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } }
I20260807 06:38:41.054730 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.054567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:56863)
0807 06:38:41.054650 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:41.054697 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 06:38:41.055326 21367 catalog_manager.cc:5495] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ecbfc604f3d4bc29742140f57361c18 (127.11.114.195). New cstate: current_term: 1 leader_uuid: "0ecbfc604f3d4bc29742140f57361c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecbfc604f3d4bc29742140f57361c18" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 } health_report { overall_health: UNKNOWN } } }
I20260807 06:38:41.077040 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.074783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.11.114.195:38629)
0807 06:38:41.074852 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:41.074854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:41.074862 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:41.074934 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:41.074935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:41.074945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:41.074985 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:41.075185 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:41.075657 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:41.076444 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:41.076627 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:41.076677 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:41.076802 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:41.076828 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:41.076864 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:41.076916 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 06:38:41.076935 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:38:41.078372 21563 raft_consensus.cc:1261] T 95b32419135e4fba9540b9806f2e22b2 P 110dec602478421ba7f0fd9d6806ab03 [term 1 FOLLOWER]: Refusing update from remote peer 0ecbfc604f3d4bc29742140f57361c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:41.078379 21757 raft_consensus.cc:1261] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 1 FOLLOWER]: Refusing update from remote peer 0ecbfc604f3d4bc29742140f57361c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:41.078642 22141 consensus_queue.cc:1035] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "110dec602478421ba7f0fd9d6806ab03" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:41.078779 22148 consensus_queue.cc:1035] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ddfc1398ad41d98c620c704c2e2ad4" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:41.079406 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.079282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:37383)
0807 06:38:41.079334 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:41.079368 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:38:41.080263 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.080151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:46091)
0807 06:38:41.080197 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:41.080227 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 06:38:41.080271 22153 mvcc.cc:205] Tried to move back new op lower bound from 7315803017534808064 to 7315803017439924224. Current Snapshot: MvccSnapshot[applied={T|T < 7315803017534808064}]
I20260807 06:38:41.080875 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.080781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:44617)
0807 06:38:41.080820 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:41.080845 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 06:38:41.081885 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.081766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.193:36151)
0807 06:38:41.081819 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:41.081846 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 06:38:41.082001 21367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:41.082098 21367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:41.082463 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.082377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.195:33987)
0807 06:38:41.082411 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:41.082435 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:38:41.082819 22157 mvcc.cc:205] Tried to move back new op lower bound from 7315803017534808064 to 7315803017439924224. Current Snapshot: MvccSnapshot[applied={T|T < 7315803017534808064}]
I20260807 06:38:41.083312 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.083204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:33755 (local address 127.11.114.194:37701)
0807 06:38:41.083246 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:41.083280 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 06:38:41.083896 21367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee5a25e82eca47e69d975cccd2148e37: Sending DeleteTablet for 3 replicas of tablet 95b32419135e4fba9540b9806f2e22b2
I20260807 06:38:41.084185 21728 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b32419135e4fba9540b9806f2e22b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:41 UTC) from {username='oryx', principal='oryx/127.11.114.252@KRBTEST.COM'} at 127.0.0.1:39746
I20260807 06:38:41.084251 21534 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b32419135e4fba9540b9806f2e22b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:41 UTC) from {username='oryx', principal='oryx/127.11.114.252@KRBTEST.COM'} at 127.0.0.1:57054
I20260807 06:38:41.084208 21725 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b32419135e4fba9540b9806f2e22b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:41 UTC) from {username='oryx', principal='oryx/127.11.114.252@KRBTEST.COM'} at 127.0.0.1:39746
I20260807 06:38:41.084527 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 21315
I20260807 06:38:41.090839 21908 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b32419135e4fba9540b9806f2e22b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:41 UTC) from {username='oryx', principal='oryx/127.11.114.252@KRBTEST.COM'} at 127.0.0.1:60940
I20260807 06:38:41.090986 22164 tablet_replica.cc:330] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: stopping tablet replica
I20260807 06:38:41.091025 21907 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b32419135e4fba9540b9806f2e22b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:41 UTC) from {username='oryx', principal='oryx/127.11.114.252@KRBTEST.COM'} at 127.0.0.1:60940
I20260807 06:38:41.091056 22164 raft_consensus.cc:2229] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:41.091107 22164 raft_consensus.cc:2258] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:41.091219 22166 tablet_replica.cc:330] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: stopping tablet replica
I20260807 06:38:41.091271 22166 raft_consensus.cc:2229] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:41.091327 22166 raft_consensus.cc:2258] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:41.092221 22166 ts_tablet_manager.cc:1905] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:41.092226 22164 ts_tablet_manager.cc:1905] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:41.093958 22166 ts_tablet_manager.cc:1918] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:41.094013 22166 log.cc:1196] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/95b32419135e4fba9540b9806f2e22b2
I20260807 06:38:41.094305 22166 ts_tablet_manager.cc:1939] T 95b32419135e4fba9540b9806f2e22b2 P 0ecbfc604f3d4bc29742140f57361c18: Deleting consensus metadata
I20260807 06:38:41.094457 22164 ts_tablet_manager.cc:1918] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:41.094529 22164 log.cc:1196] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/95b32419135e4fba9540b9806f2e22b2
I20260807 06:38:41.094662 21348 catalog_manager.cc:4841] TS 0ecbfc604f3d4bc29742140f57361c18 (127.11.114.195:38629): tablet 95b32419135e4fba9540b9806f2e22b2 (table test-table [id=8023e23df1ec47458007f4ca5e3c06e1]) successfully deleted
I20260807 06:38:41.094789 22164 ts_tablet_manager.cc:1939] T 95b32419135e4fba9540b9806f2e22b2 P 31ddfc1398ad41d98c620c704c2e2ad4: Deleting consensus metadata
W20260807 06:38:41.094971 21348 catalog_manager.cc:4808] TS 0ecbfc604f3d4bc29742140f57361c18 (127.11.114.195:38629): delete failed for tablet 95b32419135e4fba9540b9806f2e22b2 because the tablet was not found. No further retry: Not found: Tablet not found: 95b32419135e4fba9540b9806f2e22b2
I20260807 06:38:41.095252 21351 catalog_manager.cc:4841] TS 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155): tablet 95b32419135e4fba9540b9806f2e22b2 (table test-table [id=8023e23df1ec47458007f4ca5e3c06e1]) successfully deleted
W20260807 06:38:41.095340 21351 catalog_manager.cc:4808] TS 31ddfc1398ad41d98c620c704c2e2ad4 (127.11.114.194:37155): delete failed for tablet 95b32419135e4fba9540b9806f2e22b2 because the tablet was not found. No further retry: Not found: Tablet not found: 95b32419135e4fba9540b9806f2e22b2
W20260807 06:38:41.098308 21348 connection.cc:535] server connection from 127.11.114.193:58969 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:38:41.098515 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:41.098388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44647 (local address 127.0.0.1:57068)
0807 06:38:41.098446 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:41.098476 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:38:41.098546 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 21637
W20260807 06:38:41.098714 21353 catalog_manager.cc:4584] TS 110dec602478421ba7f0fd9d6806ab03 (127.11.114.193:44647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95b32419135e4fba9540b9806f2e22b2: Network error: Client connection negotiation failed: client connection to 127.11.114.193:44647: connect: Connection refused (error 111)
I20260807 06:38:41.111294 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 21821
I20260807 06:38:41.120189 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 20902
I20260807 06:38:41.130921 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 21183
2026-08-07T06:38:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6721 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:41 dist-test-slave-gkw7 krb5kdc[22170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:41 dist-test-slave-gkw7 krb5kdc[22170](info): set up 2 sockets
Aug 07 06:38:41 dist-test-slave-gkw7 krb5kdc[22170](info): commencing operation
krb5kdc: starting...
W20260807 06:38:42.372939 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:42 dist-test-slave-gkw7 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.254.keytab.
Entry for principal oryx/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.254.keytab.
I20260807 06:38:42.504987 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:37891
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.254.keytab
--principal=oryx/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:40365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:37891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:42.593748 22186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:42.593979 22186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:42.594003 22186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:42.595165 22186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:42.595201 22186 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:38:42.595214 22186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:42.595234 22186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:42.595247 22186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:42.595259 22186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:42.596746 22186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:40365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:37891
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.254.keytab
--principal=oryx/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:37891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:42.596971 22186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:42.597154 22186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:42.602453 22197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:42.602456 22193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:42.602573 22186 server_base.cc:1034] running on GCE node
W20260807 06:38:42.602453 22192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:42.603018 22186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:42.603355 22186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:42.604552 22186 hybrid_clock.cc:648] HybridClock initialized: now 1786084722604532 us; error 32 us; skew 500 ppm
Aug 07 06:38:42 dist-test-slave-gkw7 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084722, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:42.607380 22186 init.cc:377] Logged in from keytab as oryx/127.11.114.254@KRBTEST.COM (short username oryx)
I20260807 06:38:42.607940 22186 webserver.cc:460] Webserver started at http://127.11.114.254:45281/ using document root <none> and password file <none>
I20260807 06:38:42.608194 22186 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:42.608269 22186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:42.608425 22186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:42.609609 22186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "f304f17c43cc44cfb846fe9ac0234865"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "70d1d0be7015e1922d0a938bc933bbc7"
server_key_iv: "3b68d004ef62c94ab04b14aa2827e0d5"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.610000 22186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "f304f17c43cc44cfb846fe9ac0234865"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "70d1d0be7015e1922d0a938bc933bbc7"
server_key_iv: "3b68d004ef62c94ab04b14aa2827e0d5"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.611503 22186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:42.612473 22207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:42.612748 22186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:42.612874 22186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "f304f17c43cc44cfb846fe9ac0234865"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "70d1d0be7015e1922d0a938bc933bbc7"
server_key_iv: "3b68d004ef62c94ab04b14aa2827e0d5"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.612958 22186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:42.632153 22186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:42.634806 22186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:42.635023 22186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:42.640532 22186 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:37891
I20260807 06:38:42.640583 22286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:37891 every 8 connection(s)
I20260807 06:38:42.640967 22186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:42.641243 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 22186
I20260807 06:38:42.641348 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:42.641690 11723 external_mini_cluster.cc:1428] Setting key 5afbfa945a3fcbb80720b9a1e31991ed
I20260807 06:38:42.641963 22287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:42.644902 22287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: Bootstrap starting.
I20260807 06:38:42.645650 22287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:42.645962 22287 log.cc:824] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:42.646687 22287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: No bootstrap required, opened a new log
I20260807 06:38:42.648524 22287 raft_consensus.cc:348] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f304f17c43cc44cfb846fe9ac0234865" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37891 } }
I20260807 06:38:42.648689 22287 raft_consensus.cc:374] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:42.648737 22287 raft_consensus.cc:729] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f304f17c43cc44cfb846fe9ac0234865, State: Initialized, Role: FOLLOWER
I20260807 06:38:42.648876 22287 consensus_queue.cc:260] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f304f17c43cc44cfb846fe9ac0234865" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37891 } }
I20260807 06:38:42.648975 22287 raft_consensus.cc:388] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 07 06:38:42 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:42.649019 22287 raft_consensus.cc:482] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:42.649072 22287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:42.649834 22287 raft_consensus.cc:504] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f304f17c43cc44cfb846fe9ac0234865" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37891 } }
I20260807 06:38:42.649967 22287 leader_election.cc:302] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f304f17c43cc44cfb846fe9ac0234865; no voters:
I20260807 06:38:42.650142 22287 leader_election.cc:288] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:42.650208 22293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:42.650357 22293 raft_consensus.cc:686] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [term 1 LEADER]: Becoming Leader. State: Replica: f304f17c43cc44cfb846fe9ac0234865, State: Running, Role: LEADER
I20260807 06:38:42.650435 22287 sys_catalog.cc:564] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:42.650467 22293 consensus_queue.cc:237] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f304f17c43cc44cfb846fe9ac0234865" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37891 } }
I20260807 06:38:42.650835 22293 sys_catalog.cc:455] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f304f17c43cc44cfb846fe9ac0234865. Latest consensus state: current_term: 1 leader_uuid: "f304f17c43cc44cfb846fe9ac0234865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f304f17c43cc44cfb846fe9ac0234865" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37891 } } }
I20260807 06:38:42.651098 22293 sys_catalog.cc:458] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:42.651260 22295 sys_catalog.cc:455] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f304f17c43cc44cfb846fe9ac0234865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f304f17c43cc44cfb846fe9ac0234865" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 37891 } } }
I20260807 06:38:42.651336 22295 sys_catalog.cc:458] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:42.652180 22310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:42.652830 22310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:42.653930 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.642335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43118 (local address 127.11.114.254:37891)
0807 06:38:42.642559 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:42.642570 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:42.642610 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:38:42.643098 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:42.643105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:42.643164 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:42.643336 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:42.644063 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.645152 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.646222 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.646683 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.646864 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.649659 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:42.649665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:42.649665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:42.649677 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:42.652105 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.652577 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.652581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.652581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.652696 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.652957 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.652958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.652958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.653205 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:42.653390 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:42.653492 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 06:38:42.653678 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
I20260807 06:38:42.654724 22310 catalog_manager.cc:1347] Generated new cluster ID: 50a9aa173376427c99650e617d59e229
I20260807 06:38:42.654778 22310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:42.667114 22310 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:42.667927 22310 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:42.672111 22310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: Generated new TSK 0
I20260807 06:38:42.672331 22310 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab.
Entry for principal oryx/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab.
I20260807 06:38:42.704928 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37891
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab
--principal=oryx/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:40365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:42.789470 22326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:42.789773 22326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:42.789814 22326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:42.790961 22326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:42.791028 22326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:42.791061 22326 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:42.791097 22326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:42.792631 22326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:40365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab
--principal=oryx/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:37891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:42.792909 22326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:42.793152 22326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:42.798511 22331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:42.798499 22337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:42.798497 22333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:42.798743 22326 server_base.cc:1034] running on GCE node
I20260807 06:38:42.799021 22326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:42.799273 22326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:42.800462 22326 hybrid_clock.cc:648] HybridClock initialized: now 1786084722800428 us; error 33 us; skew 500 ppm
Aug 07 06:38:42 dist-test-slave-gkw7 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084722, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:42.803074 22326 init.cc:377] Logged in from keytab as oryx/127.11.114.193@KRBTEST.COM (short username oryx)
I20260807 06:38:42.803481 22326 webserver.cc:460] Webserver started at http://127.11.114.193:37233/ using document root <none> and password file <none>
I20260807 06:38:42.803699 22326 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:42.803742 22326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:42.803843 22326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:42.804876 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "ead773b5185742fcaf4e12ddc271134b"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "9c847714c69a2a949361f3d63b5594f2"
server_key_iv: "35da57e8fbc0882bf64a948a61b0f47a"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.805187 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "ead773b5185742fcaf4e12ddc271134b"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "9c847714c69a2a949361f3d63b5594f2"
server_key_iv: "35da57e8fbc0882bf64a948a61b0f47a"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.806597 22326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:42.807310 22344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:42.807500 22326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:42.807587 22326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "ead773b5185742fcaf4e12ddc271134b"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "9c847714c69a2a949361f3d63b5594f2"
server_key_iv: "35da57e8fbc0882bf64a948a61b0f47a"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.807644 22326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:42.822818 22326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:42.825245 22326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:42.825371 22326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:42.825951 22326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:42.825999 22326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:42.826025 22326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:42.826038 22326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:38:42 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084722, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:42.833271 22326 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:43229
I20260807 06:38:42.833346 22492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:43229 every 8 connection(s)
I20260807 06:38:42.833721 22326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:42.836480 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.826526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:48725 (local address 127.11.114.254:37891)
0807 06:38:42.826601 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:42.826604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:42.826817 (+ 213us) server_negotiation.cc:407] Connection header received
0807 06:38:42.827333 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:42.827334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:42.827349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:42.827430 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:42.828331 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.829218 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.830391 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.830742 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.830818 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.833650 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:42.833651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:42.833651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:42.833658 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:42.835038 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.835464 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.835465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.835465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.835535 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.835805 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.835806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.835807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.835929 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:42.836002 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:42.836296 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 06:38:42.836387 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:38:42.836673 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.826495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37891 (local address 127.11.114.193:48725)
0807 06:38:42.826674 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:42.826716 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:42.827216 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:42.827484 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:42.827488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:42.827885 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:42.828179 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:42.828186 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.829339 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.829340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:42.830290 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:42.830293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.830859 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.830860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:42.831048 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.831586 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:42.831598 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:42.833566 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:42.835141 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:42.835148 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:42.835150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:42.835377 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:42.835614 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:42.835615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:42.835616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:42.835727 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:42.836007 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:42.836015 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:42.836150 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 06:38:42.836274 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:38:42.836435 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260807 06:38:42.837149 22493 heartbeater.cc:347] Connected to a master server at 127.11.114.254:37891
I20260807 06:38:42.837256 22493 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:42.837432 22493 heartbeater.cc:511] Master 127.11.114.254:37891 requested a full tablet report, sending...
I20260807 06:38:42.838071 22230 ts_manager.cc:194] Registered new tserver with Master: ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229)
I20260807 06:38:42.839001 22230 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:48725
I20260807 06:38:42.841303 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 22326
I20260807 06:38:42.841398 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:42.841651 11723 external_mini_cluster.cc:1428] Setting key b6ae5d3eecb000beb94bd9fc117fbed8
I20260807 06:38:42.843855 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.838282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:46635 (local address 127.11.114.254:37891)
0807 06:38:42.838324 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:42.838326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:42.838471 (+ 145us) server_negotiation.cc:407] Connection header received
0807 06:38:42.838521 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:42.838522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:42.838530 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:42.838571 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:42.838892 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.839421 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.840232 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.840404 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.840450 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.841231 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:42.841233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:42.841233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:42.841249 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:42.842719 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.843071 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.843072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.843072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.843126 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.843335 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.843336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.843336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.843446 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:42.843527 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:42.843693 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 06:38:42.843766 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:38:42.843873 22500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.838237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37891 (local address 127.11.114.193:46635)
0807 06:38:42.838391 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:42.838402 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:42.838466 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:42.838583 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:42.838583 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:42.838767 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:42.838833 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:42.838834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.839483 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.839484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:42.840162 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:42.840164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.840466 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.840466 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:42.840512 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.840856 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:42.840860 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:42.841139 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:42.842824 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:42.842825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:42.842826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:42.842997 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:42.843203 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:42.843204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:42.843204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:42.843263 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:42.843518 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:42.843519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:42.843566 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 06:38:42.843664 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:38:42.843773 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab.
Entry for principal oryx/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab.
I20260807 06:38:42.868360 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37891
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab
--principal=oryx/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:40365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:42.951141 22503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:42.951418 22503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:42.951462 22503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:42.952618 22503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:42.952687 22503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:42.952731 22503 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:42.952770 22503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:42.954303 22503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:40365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab
--principal=oryx/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:37891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:42.954576 22503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:42.954826 22503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:42.959932 22513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:42.959914 22511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:42.959911 22508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:42.960222 22503 server_base.cc:1034] running on GCE node
I20260807 06:38:42.960433 22503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:42.960666 22503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:42.961831 22503 hybrid_clock.cc:648] HybridClock initialized: now 1786084722961813 us; error 44 us; skew 500 ppm
Aug 07 06:38:42 dist-test-slave-gkw7 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084722, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:42.964584 22503 init.cc:377] Logged in from keytab as oryx/127.11.114.194@KRBTEST.COM (short username oryx)
I20260807 06:38:42.965035 22503 webserver.cc:460] Webserver started at http://127.11.114.194:33401/ using document root <none> and password file <none>
I20260807 06:38:42.965270 22503 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:42.965337 22503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:42.965492 22503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:42.966581 22503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "003fafab1d3a4d2086aaa0c8423dc5f1"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "9d6824c33fab202f417ba5dea8e88258"
server_key_iv: "0f48d702138377c35405c209cae27c40"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.966965 22503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "003fafab1d3a4d2086aaa0c8423dc5f1"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "9d6824c33fab202f417ba5dea8e88258"
server_key_iv: "0f48d702138377c35405c209cae27c40"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.968374 22503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:42.969238 22521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:42.969466 22503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:42.969584 22503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "003fafab1d3a4d2086aaa0c8423dc5f1"
format_stamp: "Formatted at 2026-08-07 06:38:42 on dist-test-slave-gkw7"
server_key: "9d6824c33fab202f417ba5dea8e88258"
server_key_iv: "0f48d702138377c35405c209cae27c40"
server_key_version: "encryptionkey@0"
I20260807 06:38:42.969671 22503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:42.977672 22503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:42.979754 22503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:42.979930 22503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:42.980480 22503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:42.980545 22503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:42.980592 22503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:42.980619 22503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:38:42 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084722, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:42.988215 22503 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:46709
I20260807 06:38:42.988274 22689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:46709 every 8 connection(s)
I20260807 06:38:42.988652 22503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:42.991245 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.981094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:43035 (local address 127.11.114.254:37891)
0807 06:38:42.981177 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:42.981180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:42.981410 (+ 230us) server_negotiation.cc:407] Connection header received
0807 06:38:42.982074 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:42.982076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:42.982092 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:42.982174 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:42.983179 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.984047 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.985172 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.985515 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.985647 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.988210 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:42.988212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:42.988212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:42.988222 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:42.989884 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.990346 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.990347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.990348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.990407 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.990647 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.990648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.990648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.990769 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:42.990825 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:42.991096 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 06:38:42.991175 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:38:42.991410 22534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.981064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37891 (local address 127.11.114.194:43035)
0807 06:38:42.981266 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:42.981309 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:42.981964 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:42.982313 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:42.982318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:42.982740 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:42.983060 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:42.983068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.984164 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.984165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:42.985059 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:42.985062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.985679 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.985681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:42.985890 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.986410 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:42.986420 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:42.988073 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:42.989969 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:42.989977 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:42.989979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:42.990266 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:42.990475 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:42.990475 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:42.990476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:42.990585 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:42.990831 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:42.990836 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:42.990951 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 06:38:42.991078 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 06:38:42.991210 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":133,"threads_started":1}
I20260807 06:38:42.991961 22690 heartbeater.cc:347] Connected to a master server at 127.11.114.254:37891
I20260807 06:38:42.992049 22690 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:42.992256 22690 heartbeater.cc:511] Master 127.11.114.254:37891 requested a full tablet report, sending...
I20260807 06:38:42.992707 22231 ts_manager.cc:194] Registered new tserver with Master: 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709)
I20260807 06:38:42.993088 22231 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:43035
I20260807 06:38:42.993597 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 22503
I20260807 06:38:42.993777 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:42.994035 11723 external_mini_cluster.cc:1428] Setting key b7420ee915810a056b518ff482c2a872
I20260807 06:38:42.999815 22701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.993146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37891 (local address 127.11.114.194:40637)
0807 06:38:42.993308 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:42.993321 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:42.993386 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:42.993610 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:42.993611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:42.993823 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:42.993910 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:42.993912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.994750 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.994751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:42.995292 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:42.995294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.995817 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.995818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:42.995891 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.996453 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:42.996460 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:42.996807 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:42.998669 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:42.998670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:42.998670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:42.998852 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:42.999079 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:42.999079 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:42.999080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:42.999136 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:42.999530 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:42.999531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:42.999576 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 06:38:42.999641 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:38:42.999731 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260807 06:38:42.999894 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:42.993221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:40637 (local address 127.11.114.254:37891)
0807 06:38:42.993291 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:42.993293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:42.993376 (+ 83us) server_negotiation.cc:407] Connection header received
0807 06:38:42.993458 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:42.993459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:42.993473 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:42.993557 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:42.994006 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.994656 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.995383 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:42.995719 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:42.995803 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:42.996900 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:42.996902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:42.996902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:42.996910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:42.998580 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.998928 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.998929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.998930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.998996 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:42.999225 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:42.999226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:42.999227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:42.999443 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:42.999534 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:42.999698 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 06:38:42.999786 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for oryx/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab.
Entry for principal oryx/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab.
I20260807 06:38:43.020579 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:37891
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab
--principal=oryx/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:40365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:43.104317 22705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:43.104532 22705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:43.104555 22705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:43.105759 22705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:43.105803 22705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:43.105824 22705 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:38:43.105842 22705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:43.107316 22705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:40365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab
--principal=oryx/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.114.254:37891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:43.107499 22705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:43.107676 22705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:43.112903 22713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:43.112919 22712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:43.112916 22715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:43.113162 22705 server_base.cc:1034] running on GCE node
I20260807 06:38:43.113394 22705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:43.113656 22705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:43.114818 22705 hybrid_clock.cc:648] HybridClock initialized: now 1786084723114799 us; error 35 us; skew 500 ppm
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084723, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:43.117763 22705 init.cc:377] Logged in from keytab as oryx/127.11.114.195@KRBTEST.COM (short username oryx)
I20260807 06:38:43.118274 22705 webserver.cc:460] Webserver started at http://127.11.114.195:46587/ using document root <none> and password file <none>
I20260807 06:38:43.118530 22705 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:43.118577 22705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:43.118755 22705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:43.119833 22705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "635519f6fb3c4633bb02bd324aec8a73"
format_stamp: "Formatted at 2026-08-07 06:38:43 on dist-test-slave-gkw7"
server_key: "2f7078dd76e4b12dbd3d4a954735ad02"
server_key_iv: "08d5254a5707f33c72f1ce76408dede2"
server_key_version: "encryptionkey@0"
I20260807 06:38:43.120206 22705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "635519f6fb3c4633bb02bd324aec8a73"
format_stamp: "Formatted at 2026-08-07 06:38:43 on dist-test-slave-gkw7"
server_key: "2f7078dd76e4b12dbd3d4a954735ad02"
server_key_iv: "08d5254a5707f33c72f1ce76408dede2"
server_key_version: "encryptionkey@0"
I20260807 06:38:43.121651 22705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260807 06:38:43.122464 22723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.122682 22705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.122766 22705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "635519f6fb3c4633bb02bd324aec8a73"
format_stamp: "Formatted at 2026-08-07 06:38:43 on dist-test-slave-gkw7"
server_key: "2f7078dd76e4b12dbd3d4a954735ad02"
server_key_iv: "08d5254a5707f33c72f1ce76408dede2"
server_key_version: "encryptionkey@0"
I20260807 06:38:43.122859 22705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:43.137872 22705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:43.139986 22705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:43.140199 22705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:43.140771 22705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:43.140847 22705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.140899 22705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:43.140931 22705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.148603 22705 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:40851
I20260807 06:38:43.148671 22877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:40851 every 8 connection(s)
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084723, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:43.149076 22705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:38:43.152597 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.141313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:36869 (local address 127.11.114.254:37891)
0807 06:38:43.141393 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.141395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.141707 (+ 312us) server_negotiation.cc:407] Connection header received
0807 06:38:43.142268 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.142270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.142282 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.142336 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:43.143183 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.144002 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.146000 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.146342 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.146416 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.149383 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:43.149385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:43.149386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:43.149395 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:43.151039 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.151538 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.151539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.151540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.151624 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.151883 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.151884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.151884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.152007 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:43.152085 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.152405 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.152480 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 06:38:43.152783 22739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.141266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37891 (local address 127.11.114.195:36869)
0807 06:38:43.141582 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.141620 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.142154 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.142422 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.142428 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.142804 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:43.143095 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.143101 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.144140 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.144142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.145868 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.145873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.146449 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.146451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.146619 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.147279 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:43.147292 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:43.149289 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:43.151145 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:43.151152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:43.151155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:43.151388 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:43.151719 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:43.151720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:43.151720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:43.151821 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:43.152093 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:43.152100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:43.152279 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.152368 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.152494 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":101,"threads_started":1}
I20260807 06:38:43.153275 22878 heartbeater.cc:347] Connected to a master server at 127.11.114.254:37891
I20260807 06:38:43.153368 22878 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:43.153612 22878 heartbeater.cc:511] Master 127.11.114.254:37891 requested a full tablet report, sending...
I20260807 06:38:43.154058 22230 ts_manager.cc:194] Registered new tserver with Master: 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851)
I20260807 06:38:43.154568 22230 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:36869
I20260807 06:38:43.156354 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 22705
I20260807 06:38:43.156484 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:43.156701 11723 external_mini_cluster.cc:1428] Setting key 055a52f75cce9b07971760bf6d1f8728
I20260807 06:38:43.157742 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:38:43.161813 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.154397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37891 (local address 127.11.114.195:39065)
0807 06:38:43.154545 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.154556 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.154621 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.154757 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.154758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.155003 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:43.155094 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.155096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.155730 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.155731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.156395 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.156399 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.156852 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.156853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.156925 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.157418 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:43.157422 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:43.157856 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:43.160674 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:43.160675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:43.160675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:43.160851 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:43.161098 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:43.161099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:43.161099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:43.161151 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:43.161446 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:43.161447 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:43.161485 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.161597 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.161723 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260807 06:38:43.161902 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.154494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:39065 (local address 127.11.114.254:37891)
0807 06:38:43.154539 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.154541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.154608 (+ 67us) server_negotiation.cc:407] Connection header received
0807 06:38:43.154684 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.154685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.154697 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.154751 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:43.155182 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.155634 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.156537 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.156785 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.156851 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.157930 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:43.157937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:43.157937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:43.157943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:43.160579 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.160947 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.160949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.160949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.161025 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.161219 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.161220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.161220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.161377 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:43.161454 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.161654 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.161747 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084723, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:38:43.182320 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:43.176838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43122 (local address 127.11.114.254:37891)
0807 06:38:43.176906 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.176909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.176949 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:38:43.176973 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.176974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.176983 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.177028 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:43.177649 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.178190 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.178908 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.179105 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.179183 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.182208 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43122: BlockingRecv error: recv got EOF from 127.0.0.1:43122 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084723, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:43.189607 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.183050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.11.114.254:37891)
0807 06:38:43.183093 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.183095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.183169 (+ 74us) server_negotiation.cc:407] Connection header received
0807 06:38:43.183193 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.183194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.183202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.183249 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:43.183684 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.184132 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.184787 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.184958 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.185006 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.187200 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:43.187201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:43.187202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:43.187209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:43.188499 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.188818 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.188819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.188819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.188875 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.189044 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.189045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.189045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.189182 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:43.189270 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.189377 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.189440 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:38:43.191381 22230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:43.193241 22230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:43.199987 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.195725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43229 (local address 127.0.0.1:35462)
0807 06:38:43.195873 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.195920 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.195996 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.196343 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.196346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.196348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.196419 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.196421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.197452 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.197453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.198684 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.198687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.199650 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.199650 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.199737 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.199752 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.199834 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.199894 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":92,"threads_started":1}
I20260807 06:38:43.200002 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.195941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.11.114.193:43229)
0807 06:38:43.196143 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.196148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.196158 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:43.196226 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.196229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.196247 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.196365 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.196510 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.197370 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.198777 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.199561 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.199673 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.199742 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.199849 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.199892 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260807 06:38:43.200338 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.195833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.11.114.195:40851)
0807 06:38:43.196026 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.196030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.196245 (+ 215us) server_negotiation.cc:407] Connection header received
0807 06:38:43.196314 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.196316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.196333 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.196406 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.196645 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.197917 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.199149 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.199695 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.199798 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.199852 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.200194 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.200237 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
I20260807 06:38:43.200529 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.195728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:40851 (local address 127.0.0.1:51810)
0807 06:38:43.196143 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.196153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.196220 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.196485 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.196486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.196488 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.196553 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.196555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.198010 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.198011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.199065 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.199068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.199888 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.199889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.199954 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.199957 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.200005 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.200455 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":59,"threads_started":1}
I20260807 06:38:43.200759 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.195930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:46709 (local address 127.0.0.1:50360)
0807 06:38:43.196466 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.196478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.196533 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.196792 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.196793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.196795 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.196840 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.196841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.198194 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.198195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.199495 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.199498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.200523 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.200524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.200592 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.200596 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.200651 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.200692 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":81,"threads_started":1}
I20260807 06:38:43.201053 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.196124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50360 (local address 127.11.114.194:46709)
0807 06:38:43.196321 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.196326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.196617 (+ 291us) server_negotiation.cc:407] Connection header received
0807 06:38:43.196669 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.196673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.196691 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.196778 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.196925 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.198105 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.199597 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.200349 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.200441 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.200492 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.200913 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.200956 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":91,"threads_started":1}
I20260807 06:38:43.200981 22408 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd816f2328a45d1af7da629f81c22c6 (DEFAULT_TABLE table=test-table [id=e2ef1a99a02047cbae5622bf4795fb50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.201247 22792 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd816f2328a45d1af7da629f81c22c6 (DEFAULT_TABLE table=test-table [id=e2ef1a99a02047cbae5622bf4795fb50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.201434 22408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd816f2328a45d1af7da629f81c22c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.201647 22792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd816f2328a45d1af7da629f81c22c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.202005 22595 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd816f2328a45d1af7da629f81c22c6 (DEFAULT_TABLE table=test-table [id=e2ef1a99a02047cbae5622bf4795fb50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.202347 22595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd816f2328a45d1af7da629f81c22c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.204645 22914 tablet_bootstrap.cc:492] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: Bootstrap starting.
I20260807 06:38:43.205183 22914 tablet_bootstrap.cc:654] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.205482 22914 log.cc:824] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:43.205879 22917 tablet_bootstrap.cc:492] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Bootstrap starting.
I20260807 06:38:43.206550 22917 tablet_bootstrap.cc:654] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.206851 22917 log.cc:824] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:43.207695 22914 tablet_bootstrap.cc:492] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: No bootstrap required, opened a new log
I20260807 06:38:43.207782 22914 ts_tablet_manager.cc:1397] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:38:43.209846 22917 tablet_bootstrap.cc:492] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: No bootstrap required, opened a new log
I20260807 06:38:43.209933 22917 ts_tablet_manager.cc:1397] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:38:43.210142 22919 tablet_bootstrap.cc:492] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Bootstrap starting.
I20260807 06:38:43.210412 22914 raft_consensus.cc:348] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.210572 22914 raft_consensus.cc:374] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.210644 22914 raft_consensus.cc:729] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635519f6fb3c4633bb02bd324aec8a73, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.210777 22914 consensus_queue.cc:260] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.210932 22919 tablet_bootstrap.cc:654] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.211094 22914 ts_tablet_manager.cc:1428] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 06:38:43.211248 22878 heartbeater.cc:503] Master 127.11.114.254:37891 was elected leader, sending a full tablet report...
I20260807 06:38:43.211323 22919 log.cc:824] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:43.212184 22919 tablet_bootstrap.cc:492] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: No bootstrap required, opened a new log
I20260807 06:38:43.212272 22919 ts_tablet_manager.cc:1397] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:43.212134 22917 raft_consensus.cc:348] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.212349 22917 raft_consensus.cc:374] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.212405 22917 raft_consensus.cc:729] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003fafab1d3a4d2086aaa0c8423dc5f1, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.212518 22917 consensus_queue.cc:260] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.212786 22917 ts_tablet_manager.cc:1428] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:38:43.212942 22690 heartbeater.cc:503] Master 127.11.114.254:37891 was elected leader, sending a full tablet report...
I20260807 06:38:43.214430 22919 raft_consensus.cc:348] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.214602 22919 raft_consensus.cc:374] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.214655 22919 raft_consensus.cc:729] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead773b5185742fcaf4e12ddc271134b, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.214807 22919 consensus_queue.cc:260] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.215097 22919 ts_tablet_manager.cc:1428] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:38:43.215152 22493 heartbeater.cc:503] Master 127.11.114.254:37891 was elected leader, sending a full tablet report...
W20260807 06:38:43.239279 22695 tablet.cc:2369] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:38:43.334692 22494 tablet.cc:2369] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:38:43.359282 22925 raft_consensus.cc:482] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:43.359402 22925 raft_consensus.cc:504] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.359781 22925 leader_election.cc:288] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851), 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709)
I20260807 06:38:43.362877 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.360030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:40851 (local address 127.11.114.193:36239)
0807 06:38:43.360179 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.360193 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.360269 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.360485 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.360487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.360489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.360560 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.360561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.361199 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.361200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.362049 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.362052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.362593 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.362594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.362664 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.362668 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.362751 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.362794 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260807 06:38:43.362866 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.360123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:36239 (local address 127.11.114.195:40851)
0807 06:38:43.360188 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.360190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.360265 (+ 75us) server_negotiation.cc:407] Connection header received
0807 06:38:43.360336 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.360337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.360349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.360432 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.360634 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.361117 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.362143 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.362505 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.362595 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.362604 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.362760 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.362798 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 06:38:43.363170 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.360248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:46709 (local address 127.11.114.193:34985)
0807 06:38:43.360382 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.360393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.360444 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.360603 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.360604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.360607 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.360692 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.360694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.361433 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.361434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.362266 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.362268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.362874 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.362876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.362962 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.362967 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.363031 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.363085 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260807 06:38:43.363184 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.360366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:34985 (local address 127.11.114.194:46709)
0807 06:38:43.360430 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.360433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.360444 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:38:43.360504 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.360505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.360517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.360596 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.360772 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.361335 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.362370 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.362772 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.362861 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.362870 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.363080 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.363127 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 06:38:43.363183 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd816f2328a45d1af7da629f81c22c6" candidate_uuid: "ead773b5185742fcaf4e12ddc271134b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635519f6fb3c4633bb02bd324aec8a73" is_pre_election: true
I20260807 06:38:43.363371 22817 raft_consensus.cc:2454] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead773b5185742fcaf4e12ddc271134b in term 0.
I20260807 06:38:43.363509 22626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd816f2328a45d1af7da629f81c22c6" candidate_uuid: "ead773b5185742fcaf4e12ddc271134b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" is_pre_election: true
I20260807 06:38:43.363654 22346 leader_election.cc:302] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 635519f6fb3c4633bb02bd324aec8a73, ead773b5185742fcaf4e12ddc271134b; no voters:
I20260807 06:38:43.363678 22626 raft_consensus.cc:2454] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead773b5185742fcaf4e12ddc271134b in term 0.
I20260807 06:38:43.363788 22925 raft_consensus.cc:2781] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:43.363865 22925 raft_consensus.cc:482] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:43.363896 22925 raft_consensus.cc:3037] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.364632 22925 raft_consensus.cc:504] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.364804 22925 leader_election.cc:288] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [CANDIDATE]: Term 1 election: Requested vote from peers 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851), 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709)
I20260807 06:38:43.365041 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd816f2328a45d1af7da629f81c22c6" candidate_uuid: "ead773b5185742fcaf4e12ddc271134b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635519f6fb3c4633bb02bd324aec8a73"
I20260807 06:38:43.365083 22626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd816f2328a45d1af7da629f81c22c6" candidate_uuid: "ead773b5185742fcaf4e12ddc271134b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1"
I20260807 06:38:43.365134 22817 raft_consensus.cc:3037] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.365204 22626 raft_consensus.cc:3037] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.365935 22817 raft_consensus.cc:2454] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead773b5185742fcaf4e12ddc271134b in term 1.
I20260807 06:38:43.366016 22626 raft_consensus.cc:2454] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead773b5185742fcaf4e12ddc271134b in term 1.
I20260807 06:38:43.366154 22346 leader_election.cc:302] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 635519f6fb3c4633bb02bd324aec8a73, ead773b5185742fcaf4e12ddc271134b; no voters:
I20260807 06:38:43.366287 22925 raft_consensus.cc:2781] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:43.366431 22925 raft_consensus.cc:686] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 1 LEADER]: Becoming Leader. State: Replica: ead773b5185742fcaf4e12ddc271134b, State: Running, Role: LEADER
I20260807 06:38:43.366524 22925 consensus_queue.cc:237] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } }
I20260807 06:38:43.367344 22230 catalog_manager.cc:5495] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b reported cstate change: term changed from 0 to 1, leader changed from <none> to ead773b5185742fcaf4e12ddc271134b (127.11.114.193). New cstate: current_term: 1 leader_uuid: "ead773b5185742fcaf4e12ddc271134b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } health_report { overall_health: UNKNOWN } } }
I20260807 06:38:43.385878 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.383429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.11.114.193:43229)
0807 06:38:43.383503 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.383505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.383515 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:43.383552 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.383553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.383573 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.383662 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:43.383822 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.384289 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.385034 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.385247 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.385321 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.385508 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.385650 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.385756 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.385773 (+ 17us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.385793 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 06:38:43.387786 22626 raft_consensus.cc:1261] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Refusing update from remote peer ead773b5185742fcaf4e12ddc271134b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.387786 22817 raft_consensus.cc:1261] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Refusing update from remote peer ead773b5185742fcaf4e12ddc271134b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.388195 22931 consensus_queue.cc:1035] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [LEADER]: Connected to new peer: Peer: permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.388386 22925 consensus_queue.cc:1035] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [LEADER]: Connected to new peer: Peer: permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.391258 22944 mvcc.cc:205] Tried to move back new op lower bound from 7315803026993496064 to 7315803026909892608. Current Snapshot: MvccSnapshot[applied={T|T < 7315803026993496064}]
I20260807 06:38:43.392067 22942 mvcc.cc:205] Tried to move back new op lower bound from 7315803026993496064 to 7315803026909892608. Current Snapshot: MvccSnapshot[applied={T|T < 7315803026993496064 or (T in {7315803026993496064})}]
I20260807 06:38:43.393177 22230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:43.393272 22230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:43.394099 22230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: Sending DeleteTablet for 3 replicas of tablet 3bd816f2328a45d1af7da629f81c22c6
I20260807 06:38:43.394440 22408 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd816f2328a45d1af7da629f81c22c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:35462
I20260807 06:38:43.394495 22792 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd816f2328a45d1af7da629f81c22c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:51810
I20260807 06:38:43.394694 22595 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd816f2328a45d1af7da629f81c22c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:50360
I20260807 06:38:43.394749 22954 tablet_replica.cc:330] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: stopping tablet replica
I20260807 06:38:43.394852 22954 raft_consensus.cc:2229] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:43.395008 22954 raft_consensus.cc:2258] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.395265 22230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:43.395406 22230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:43.395380 22956 tablet_replica.cc:330] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: stopping tablet replica
I20260807 06:38:43.395630 22954 ts_tablet_manager.cc:1905] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.395946 22956 raft_consensus.cc:2229] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:43.396135 22956 raft_consensus.cc:2258] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.396705 22955 tablet_replica.cc:330] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: stopping tablet replica
I20260807 06:38:43.396746 22956 ts_tablet_manager.cc:1905] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.396862 22955 raft_consensus.cc:2229] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:43.396924 22408 tablet_service.cc:1467] Processing CreateTablet for tablet 16595af5ab2b4bd19e308c65bb4e96b2 (DEFAULT_TABLE table=test-table [id=97510b6d107f4a0f881aaf6f0e906562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.397027 22955 raft_consensus.cc:2258] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.397043 22408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16595af5ab2b4bd19e308c65bb4e96b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.397761 22595 tablet_service.cc:1467] Processing CreateTablet for tablet 16595af5ab2b4bd19e308c65bb4e96b2 (DEFAULT_TABLE table=test-table [id=97510b6d107f4a0f881aaf6f0e906562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.397888 22595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16595af5ab2b4bd19e308c65bb4e96b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.398597 22956 ts_tablet_manager.cc:1918] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:43.398665 22956 log.cc:1196] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/3bd816f2328a45d1af7da629f81c22c6
I20260807 06:38:43.398942 22956 ts_tablet_manager.cc:1939] T 3bd816f2328a45d1af7da629f81c22c6 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting consensus metadata
I20260807 06:38:43.399323 22917 tablet_bootstrap.cc:492] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: Bootstrap starting.
I20260807 06:38:43.399351 22919 tablet_bootstrap.cc:492] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: Bootstrap starting.
I20260807 06:38:43.399724 22917 tablet_bootstrap.cc:654] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.399829 22919 tablet_bootstrap.cc:654] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.400331 22209 catalog_manager.cc:4841] TS 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709): tablet 3bd816f2328a45d1af7da629f81c22c6 (table test-table [id=e2ef1a99a02047cbae5622bf4795fb50]) successfully deleted
I20260807 06:38:43.400494 22919 tablet_bootstrap.cc:492] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: No bootstrap required, opened a new log
I20260807 06:38:43.400547 22919 ts_tablet_manager.cc:1397] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.400619 22954 ts_tablet_manager.cc:1918] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:43.400691 22954 log.cc:1196] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/3bd816f2328a45d1af7da629f81c22c6
I20260807 06:38:43.400688 22919 raft_consensus.cc:348] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.400806 22919 raft_consensus.cc:374] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.400797 22792 tablet_service.cc:1467] Processing CreateTablet for tablet 16595af5ab2b4bd19e308c65bb4e96b2 (DEFAULT_TABLE table=test-table [id=97510b6d107f4a0f881aaf6f0e906562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.400833 22919 raft_consensus.cc:729] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead773b5185742fcaf4e12ddc271134b, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.400915 22792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16595af5ab2b4bd19e308c65bb4e96b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.400883 22919 consensus_queue.cc:260] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.401031 22919 ts_tablet_manager.cc:1428] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.401503 22954 ts_tablet_manager.cc:1939] T 3bd816f2328a45d1af7da629f81c22c6 P ead773b5185742fcaf4e12ddc271134b: Deleting consensus metadata
I20260807 06:38:43.402074 22208 catalog_manager.cc:4841] TS ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229): tablet 3bd816f2328a45d1af7da629f81c22c6 (table test-table [id=e2ef1a99a02047cbae5622bf4795fb50]) successfully deleted
I20260807 06:38:43.402798 22914 tablet_bootstrap.cc:492] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: Bootstrap starting.
I20260807 06:38:43.403251 22955 ts_tablet_manager.cc:1905] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.403280 22914 tablet_bootstrap.cc:654] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.403985 22914 tablet_bootstrap.cc:492] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: No bootstrap required, opened a new log
I20260807 06:38:43.404052 22914 ts_tablet_manager.cc:1397] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.404222 22914 raft_consensus.cc:348] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.404305 22914 raft_consensus.cc:374] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.404330 22914 raft_consensus.cc:729] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635519f6fb3c4633bb02bd324aec8a73, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.404378 22914 consensus_queue.cc:260] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.404474 22914 ts_tablet_manager.cc:1428] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.404934 22955 ts_tablet_manager.cc:1918] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:43.405009 22955 log.cc:1196] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/3bd816f2328a45d1af7da629f81c22c6
I20260807 06:38:43.405316 22955 ts_tablet_manager.cc:1939] T 3bd816f2328a45d1af7da629f81c22c6 P 635519f6fb3c4633bb02bd324aec8a73: Deleting consensus metadata
I20260807 06:38:43.405805 22209 catalog_manager.cc:4841] TS 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851): tablet 3bd816f2328a45d1af7da629f81c22c6 (table test-table [id=e2ef1a99a02047cbae5622bf4795fb50]) successfully deleted
I20260807 06:38:43.406145 22917 tablet_bootstrap.cc:492] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: No bootstrap required, opened a new log
I20260807 06:38:43.406200 22917 ts_tablet_manager.cc:1397] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260807 06:38:43.406347 22917 raft_consensus.cc:348] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.406414 22917 raft_consensus.cc:374] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.406433 22917 raft_consensus.cc:729] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003fafab1d3a4d2086aaa0c8423dc5f1, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.406476 22917 consensus_queue.cc:260] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.406560 22917 ts_tablet_manager.cc:1428] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.595600 22924 raft_consensus.cc:482] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:43.595705 22924 raft_consensus.cc:504] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.596134 22924 leader_election.cc:288] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229), 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851)
I20260807 06:38:43.599185 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.596453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:53095 (local address 127.11.114.193:43229)
0807 06:38:43.596505 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.596508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.596562 (+ 54us) server_negotiation.cc:407] Connection header received
0807 06:38:43.596652 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.596653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.596665 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.596739 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.596919 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.597393 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.598455 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.598819 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.598902 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.598912 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.599067 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.599107 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:38:43.599185 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.596429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:35929 (local address 127.11.114.195:40851)
0807 06:38:43.596497 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.596500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.596582 (+ 82us) server_negotiation.cc:407] Connection header received
0807 06:38:43.596677 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.596679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.596688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.596739 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.596899 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.597391 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.598455 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.598817 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.598911 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.598920 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.599075 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.599113 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:38:43.599213 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.596346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43229 (local address 127.11.114.194:53095)
0807 06:38:43.596495 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.596506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.596576 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.596746 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.596747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.596750 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.596839 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.596841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.597488 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.597489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.598356 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.598358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.598912 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.598913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.598992 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.598996 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.599061 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.599107 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":49,"thread_start_us":67,"threads_started":1}
I20260807 06:38:43.599215 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.596346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:40851 (local address 127.11.114.194:35929)
0807 06:38:43.596493 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.596506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.596600 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.596751 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.596752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.596754 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.596832 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.596834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.597482 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.597483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.598352 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.598355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.598914 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.598915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.598996 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.598999 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.599068 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.599110 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":20,"thread_start_us":88,"threads_started":1}
I20260807 06:38:43.599668 22432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16595af5ab2b4bd19e308c65bb4e96b2" candidate_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead773b5185742fcaf4e12ddc271134b" is_pre_election: true
I20260807 06:38:43.599846 22432 raft_consensus.cc:2454] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 003fafab1d3a4d2086aaa0c8423dc5f1 in term 0.
I20260807 06:38:43.599855 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16595af5ab2b4bd19e308c65bb4e96b2" candidate_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635519f6fb3c4633bb02bd324aec8a73" is_pre_election: true
I20260807 06:38:43.599957 22817 raft_consensus.cc:2454] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 003fafab1d3a4d2086aaa0c8423dc5f1 in term 0.
I20260807 06:38:43.600060 22523 leader_election.cc:302] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003fafab1d3a4d2086aaa0c8423dc5f1, ead773b5185742fcaf4e12ddc271134b; no voters:
I20260807 06:38:43.600210 22924 raft_consensus.cc:2781] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:43.600260 22924 raft_consensus.cc:482] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:43.600283 22924 raft_consensus.cc:3037] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.600984 22924 raft_consensus.cc:504] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.601138 22924 leader_election.cc:288] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [CANDIDATE]: Term 1 election: Requested vote from peers ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229), 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851)
I20260807 06:38:43.601284 22432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16595af5ab2b4bd19e308c65bb4e96b2" candidate_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead773b5185742fcaf4e12ddc271134b"
I20260807 06:38:43.601290 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16595af5ab2b4bd19e308c65bb4e96b2" candidate_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635519f6fb3c4633bb02bd324aec8a73"
I20260807 06:38:43.601361 22817 raft_consensus.cc:3037] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.601361 22432 raft_consensus.cc:3037] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.602206 22817 raft_consensus.cc:2454] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 003fafab1d3a4d2086aaa0c8423dc5f1 in term 1.
I20260807 06:38:43.602208 22432 raft_consensus.cc:2454] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 003fafab1d3a4d2086aaa0c8423dc5f1 in term 1.
I20260807 06:38:43.602384 22525 leader_election.cc:302] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003fafab1d3a4d2086aaa0c8423dc5f1, 635519f6fb3c4633bb02bd324aec8a73; no voters:
I20260807 06:38:43.602502 22924 raft_consensus.cc:2781] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:43.602660 22924 raft_consensus.cc:686] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 LEADER]: Becoming Leader. State: Replica: 003fafab1d3a4d2086aaa0c8423dc5f1, State: Running, Role: LEADER
I20260807 06:38:43.602751 22924 consensus_queue.cc:237] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.603479 22230 catalog_manager.cc:5495] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } health_report { overall_health: UNKNOWN } } }
W20260807 06:38:43.650369 22879 tablet.cc:2369] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084722, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.254@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:43.681068 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.674422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.11.114.254:37891)
0807 06:38:43.674512 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.674514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.674566 (+ 52us) server_negotiation.cc:407] Connection header received
0807 06:38:43.674615 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.674616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.674624 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.674678 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:43.675141 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.675633 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.676322 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.676494 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.676542 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.678620 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:43.678622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:43.678622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:43.678628 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:43.680049 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.680432 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.680432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.680433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.680484 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.680661 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.680662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.680662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.680769 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:43.680819 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.680937 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.681004 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:38:43.681182 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.674335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:37891 (local address 127.0.0.1:43150)
0807 06:38:43.674491 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.674501 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.674569 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.674700 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.674701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.675024 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:43.675077 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.675080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.675702 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.675703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.676255 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.676257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.676554 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.676555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.676596 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.677089 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:43.677094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:43.678543 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:43.680130 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:43.680137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:43.680139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:43.680365 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:43.680541 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:43.680542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:43.680542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:43.680602 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:43.680822 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:43.680828 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:43.680871 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.680922 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.681072 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":57,"thread_start_us":70,"threads_started":1}
I20260807 06:38:43.683349 22230 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:43150:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:38:43.683576 22230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:43.685212 22595 tablet_service.cc:1467] Processing CreateTablet for tablet 017053c309ad4cc8a36a8b6b4fb9d604 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d01eea51bc684ce28f3050743f5df103]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:43.685225 22408 tablet_service.cc:1467] Processing CreateTablet for tablet 017053c309ad4cc8a36a8b6b4fb9d604 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d01eea51bc684ce28f3050743f5df103]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:43.685264 22792 tablet_service.cc:1467] Processing CreateTablet for tablet 017053c309ad4cc8a36a8b6b4fb9d604 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d01eea51bc684ce28f3050743f5df103]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:38:43.685386 22595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017053c309ad4cc8a36a8b6b4fb9d604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.685386 22408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017053c309ad4cc8a36a8b6b4fb9d604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.685425 22792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017053c309ad4cc8a36a8b6b4fb9d604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.686759 22914 tablet_bootstrap.cc:492] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: Bootstrap starting.
I20260807 06:38:43.686911 22919 tablet_bootstrap.cc:492] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: Bootstrap starting.
I20260807 06:38:43.686939 22917 tablet_bootstrap.cc:492] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: Bootstrap starting.
I20260807 06:38:43.687196 22914 tablet_bootstrap.cc:654] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.687259 22919 tablet_bootstrap.cc:654] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.687376 22917 tablet_bootstrap.cc:654] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.687808 22919 tablet_bootstrap.cc:492] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: No bootstrap required, opened a new log
I20260807 06:38:43.687878 22919 ts_tablet_manager.cc:1397] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.687996 22919 raft_consensus.cc:348] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.688015 22917 tablet_bootstrap.cc:492] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: No bootstrap required, opened a new log
I20260807 06:38:43.688028 22914 tablet_bootstrap.cc:492] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: No bootstrap required, opened a new log
I20260807 06:38:43.688046 22919 raft_consensus.cc:374] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.688060 22919 raft_consensus.cc:729] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead773b5185742fcaf4e12ddc271134b, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.688064 22917 ts_tablet_manager.cc:1397] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.688066 22914 ts_tablet_manager.cc:1397] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.688131 22919 consensus_queue.cc:260] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.688225 22919 ts_tablet_manager.cc:1428] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.688212 22914 raft_consensus.cc:348] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.688211 22917 raft_consensus.cc:348] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.688272 22914 raft_consensus.cc:374] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.688284 22917 raft_consensus.cc:374] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.688241 22931 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.688297 22914 raft_consensus.cc:729] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635519f6fb3c4633bb02bd324aec8a73, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.688308 22917 raft_consensus.cc:729] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003fafab1d3a4d2086aaa0c8423dc5f1, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.688319 22931 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:43.688345 22914 consensus_queue.cc:260] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.688352 22917 consensus_queue.cc:260] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.688431 22914 ts_tablet_manager.cc:1428] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.688431 22917 ts_tablet_manager.cc:1428] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.688582 22924 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.688668 22924 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:43.688829 22920 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.688917 22920 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:43.710799 22924 raft_consensus.cc:482] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:43.710933 22924 raft_consensus.cc:504] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.711107 22924 leader_election.cc:288] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229), 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851)
I20260807 06:38:43.711339 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017053c309ad4cc8a36a8b6b4fb9d604" candidate_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635519f6fb3c4633bb02bd324aec8a73" is_pre_election: true
I20260807 06:38:43.711352 22432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017053c309ad4cc8a36a8b6b4fb9d604" candidate_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead773b5185742fcaf4e12ddc271134b" is_pre_election: true
I20260807 06:38:43.711441 22817 raft_consensus.cc:2454] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 003fafab1d3a4d2086aaa0c8423dc5f1 in term 0.
I20260807 06:38:43.711441 22432 raft_consensus.cc:2454] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 003fafab1d3a4d2086aaa0c8423dc5f1 in term 0.
I20260807 06:38:43.711591 22525 leader_election.cc:302] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003fafab1d3a4d2086aaa0c8423dc5f1, 635519f6fb3c4633bb02bd324aec8a73; no voters:
I20260807 06:38:43.711688 22924 raft_consensus.cc:2781] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:43.711731 22924 raft_consensus.cc:482] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:43.711752 22924 raft_consensus.cc:3037] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.712415 22924 raft_consensus.cc:504] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.712550 22924 leader_election.cc:288] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [CANDIDATE]: Term 1 election: Requested vote from peers ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229), 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851)
I20260807 06:38:43.712746 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017053c309ad4cc8a36a8b6b4fb9d604" candidate_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635519f6fb3c4633bb02bd324aec8a73"
I20260807 06:38:43.712735 22432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017053c309ad4cc8a36a8b6b4fb9d604" candidate_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead773b5185742fcaf4e12ddc271134b"
I20260807 06:38:43.712814 22432 raft_consensus.cc:3037] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.712814 22817 raft_consensus.cc:3037] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.713570 22432 raft_consensus.cc:2454] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 003fafab1d3a4d2086aaa0c8423dc5f1 in term 1.
I20260807 06:38:43.713584 22817 raft_consensus.cc:2454] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 003fafab1d3a4d2086aaa0c8423dc5f1 in term 1.
I20260807 06:38:43.713764 22525 leader_election.cc:302] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003fafab1d3a4d2086aaa0c8423dc5f1, 635519f6fb3c4633bb02bd324aec8a73; no voters:
I20260807 06:38:43.713852 22924 raft_consensus.cc:2781] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:43.713896 22924 raft_consensus.cc:686] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 LEADER]: Becoming Leader. State: Replica: 003fafab1d3a4d2086aaa0c8423dc5f1, State: Running, Role: LEADER
I20260807 06:38:43.713943 22924 consensus_queue.cc:237] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.714267 22965 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: TxnStatusTablet state changed. Reason: New leader 003fafab1d3a4d2086aaa0c8423dc5f1. Latest consensus state: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.714370 22965 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:43.714570 22984 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:38:43.714540 22230 catalog_manager.cc:5495] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260807 06:38:43.714685 22984 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:38:43.727164 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.724141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.11.114.194:46709)
0807 06:38:43.724183 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.724186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.724281 (+ 95us) server_negotiation.cc:407] Connection header received
0807 06:38:43.724363 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.724364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.724379 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.724434 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:43.724582 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.725067 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.726296 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.726554 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.726614 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.726816 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.726909 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.727003 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.727067 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.727090 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:38:43.727233 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.724069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:46709 (local address 127.0.0.1:50368)
0807 06:38:43.724213 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.724226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.724286 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.724460 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.724461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.724463 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:43.724516 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.724518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.725160 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.725162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.726207 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.726210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.726631 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.726632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.726695 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.726715 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.726990 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.727003 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.727062 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.727116 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260807 06:38:43.728322 22817 raft_consensus.cc:1261] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Refusing update from remote peer 003fafab1d3a4d2086aaa0c8423dc5f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.728395 22432 raft_consensus.cc:1261] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Refusing update from remote peer 003fafab1d3a4d2086aaa0c8423dc5f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.728710 22965 consensus_queue.cc:1035] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.728870 22924 consensus_queue.cc:1035] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.729393 22920 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: TxnStatusTablet state changed. Reason: New leader 003fafab1d3a4d2086aaa0c8423dc5f1. Latest consensus state: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.729492 22920 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:43.729507 22931 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: TxnStatusTablet state changed. Reason: New leader 003fafab1d3a4d2086aaa0c8423dc5f1. Latest consensus state: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.729614 22931 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:43.730329 22965 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.730405 22965 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:43.730453 22965 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.730491 22965 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P 003fafab1d3a4d2086aaa0c8423dc5f1: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:38:43.730839 22920 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.731042 22920 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P 635519f6fb3c4633bb02bd324aec8a73: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:43.731583 22941 mvcc.cc:205] Tried to move back new op lower bound from 7315803028389228544 to 7315803028332732416. Current Snapshot: MvccSnapshot[applied={T|T < 7315803028389228544}]
I20260807 06:38:43.731706 22931 tablet_replica.cc:439] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } } }
I20260807 06:38:43.731775 22931 tablet_replica.cc:442] T 017053c309ad4cc8a36a8b6b4fb9d604 P ead773b5185742fcaf4e12ddc271134b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:38:43.734612 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.731932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.11.114.194:46709)
0807 06:38:43.731977 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.731979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.732100 (+ 121us) server_negotiation.cc:407] Connection header received
0807 06:38:43.732135 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.732135 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.732145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.732192 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:43.732382 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.733105 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.733909 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.734210 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.734281 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.734388 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.734420 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.734469 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.734521 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.734544 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 06:38:43.739977 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.737255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:40993 (local address 127.11.114.194:46709)
0807 06:38:43.737293 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.737295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.737427 (+ 132us) server_negotiation.cc:407] Connection header received
0807 06:38:43.737466 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.737466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.737477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.737553 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:43.737722 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.738467 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.739193 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.739484 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.739553 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.739693 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.739725 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.739778 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.739824 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.739882 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":17}
I20260807 06:38:43.740059 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.737177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:46709 (local address 127.11.114.194:40993)
0807 06:38:43.737363 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.737372 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.737424 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.737603 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.737605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.737610 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:43.737658 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.737660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.738533 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.738534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.739106 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.739107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.739558 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.739559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.739604 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.739617 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.739785 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.739786 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.739821 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.739895 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260807 06:38:43.742043 22817 raft_consensus.cc:1261] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Refusing update from remote peer 003fafab1d3a4d2086aaa0c8423dc5f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.742187 22432 raft_consensus.cc:1261] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Refusing update from remote peer 003fafab1d3a4d2086aaa0c8423dc5f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.742244 22991 consensus_queue.cc:1035] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.742345 22924 consensus_queue.cc:1035] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.756685 22231 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:43.756791 22231 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:43.757445 22231 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: Sending DeleteTablet for 3 replicas of tablet 16595af5ab2b4bd19e308c65bb4e96b2
I20260807 06:38:43.757766 22408 tablet_service.cc:1514] Processing DeleteTablet for tablet 16595af5ab2b4bd19e308c65bb4e96b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:35462
I20260807 06:38:43.757890 22792 tablet_service.cc:1514] Processing DeleteTablet for tablet 16595af5ab2b4bd19e308c65bb4e96b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:51810
I20260807 06:38:43.757877 22954 tablet_replica.cc:330] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: stopping tablet replica
I20260807 06:38:43.757967 22954 raft_consensus.cc:2229] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:43.757974 22955 tablet_replica.cc:330] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: stopping tablet replica
I20260807 06:38:43.758006 22954 raft_consensus.cc:2258] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.758016 22955 raft_consensus.cc:2229] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:43.758051 22955 raft_consensus.cc:2258] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.758472 22595 tablet_service.cc:1514] Processing DeleteTablet for tablet 16595af5ab2b4bd19e308c65bb4e96b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:50360
I20260807 06:38:43.758554 22956 tablet_replica.cc:330] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: stopping tablet replica
I20260807 06:38:43.758596 22956 raft_consensus.cc:2229] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:43.758672 22956 raft_consensus.cc:2258] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.758970 22956 ts_tablet_manager.cc:1905] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.760569 22956 ts_tablet_manager.cc:1918] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:43.760622 22956 log.cc:1196] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/16595af5ab2b4bd19e308c65bb4e96b2
I20260807 06:38:43.760836 22956 ts_tablet_manager.cc:1939] T 16595af5ab2b4bd19e308c65bb4e96b2 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting consensus metadata
I20260807 06:38:43.761181 22209 catalog_manager.cc:4841] TS 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709): tablet 16595af5ab2b4bd19e308c65bb4e96b2 (table test-table [id=97510b6d107f4a0f881aaf6f0e906562]) successfully deleted
I20260807 06:38:43.761215 22955 ts_tablet_manager.cc:1905] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.762708 22955 ts_tablet_manager.cc:1918] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:43.762773 22955 log.cc:1196] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/16595af5ab2b4bd19e308c65bb4e96b2
I20260807 06:38:43.762979 22955 ts_tablet_manager.cc:1939] T 16595af5ab2b4bd19e308c65bb4e96b2 P 635519f6fb3c4633bb02bd324aec8a73: Deleting consensus metadata
I20260807 06:38:43.763298 22209 catalog_manager.cc:4841] TS 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851): tablet 16595af5ab2b4bd19e308c65bb4e96b2 (table test-table [id=97510b6d107f4a0f881aaf6f0e906562]) successfully deleted
I20260807 06:38:43.768337 22954 ts_tablet_manager.cc:1905] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.769775 22954 ts_tablet_manager.cc:1918] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:43.769851 22954 log.cc:1196] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/16595af5ab2b4bd19e308c65bb4e96b2
I20260807 06:38:43.770053 22954 ts_tablet_manager.cc:1939] T 16595af5ab2b4bd19e308c65bb4e96b2 P ead773b5185742fcaf4e12ddc271134b: Deleting consensus metadata
I20260807 06:38:43.770391 22208 catalog_manager.cc:4841] TS ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229): tablet 16595af5ab2b4bd19e308c65bb4e96b2 (table test-table [id=97510b6d107f4a0f881aaf6f0e906562]) successfully deleted
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:38:43.785071 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:43.779016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.11.114.254:37891)
0807 06:38:43.779069 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.779071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.779159 (+ 88us) server_negotiation.cc:407] Connection header received
0807 06:38:43.779227 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.779227 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.779237 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.779286 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:43.780031 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.780517 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.781615 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.781827 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.781879 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.784996 (+ 3117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43152: BlockingRecv error: recv got EOF from 127.0.0.1:43152 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 07 06:38:43 dist-test-slave-gkw7 krb5kdc[22170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM
I20260807 06:38:43.793278 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.785998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.11.114.254:37891)
0807 06:38:43.786036 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.786038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.786069 (+ 31us) server_negotiation.cc:407] Connection header received
0807 06:38:43.786148 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.786149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.786156 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.786200 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:43.786759 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.787210 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.788180 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.788351 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.788395 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.790702 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:43.790703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:43.790704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:43.790712 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:43.792131 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.792497 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.792499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.792499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.792560 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:43.792776 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:43.792777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:43.792777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:43.792889 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:43.792992 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.793100 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.793178 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:38:43.795037 22231 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:43.795233 22231 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:43.796746 22595 tablet_service.cc:1467] Processing CreateTablet for tablet 822af26209794cf9b76fe718eef6ed1a (DEFAULT_TABLE table=test-table [id=4ae72b5dbf134d318a823913734dfe92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.796764 22408 tablet_service.cc:1467] Processing CreateTablet for tablet 822af26209794cf9b76fe718eef6ed1a (DEFAULT_TABLE table=test-table [id=4ae72b5dbf134d318a823913734dfe92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.796902 22595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 822af26209794cf9b76fe718eef6ed1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.796917 22408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 822af26209794cf9b76fe718eef6ed1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.797992 22919 tablet_bootstrap.cc:492] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: Bootstrap starting.
I20260807 06:38:43.798259 22917 tablet_bootstrap.cc:492] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: Bootstrap starting.
I20260807 06:38:43.798348 22919 tablet_bootstrap.cc:654] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.798326 22792 tablet_service.cc:1467] Processing CreateTablet for tablet 822af26209794cf9b76fe718eef6ed1a (DEFAULT_TABLE table=test-table [id=4ae72b5dbf134d318a823913734dfe92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.798446 22792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 822af26209794cf9b76fe718eef6ed1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.798563 22917 tablet_bootstrap.cc:654] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.799078 22917 tablet_bootstrap.cc:492] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: No bootstrap required, opened a new log
I20260807 06:38:43.799113 22917 ts_tablet_manager.cc:1397] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.799193 22917 raft_consensus.cc:348] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.799248 22917 raft_consensus.cc:374] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.799263 22917 raft_consensus.cc:729] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003fafab1d3a4d2086aaa0c8423dc5f1, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.799284 22917 consensus_queue.cc:260] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.799381 22917 ts_tablet_manager.cc:1428] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.799744 22914 tablet_bootstrap.cc:492] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: Bootstrap starting.
I20260807 06:38:43.800173 22914 tablet_bootstrap.cc:654] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.800839 22919 tablet_bootstrap.cc:492] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: No bootstrap required, opened a new log
I20260807 06:38:43.800895 22919 ts_tablet_manager.cc:1397] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:38:43.800987 22914 tablet_bootstrap.cc:492] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: No bootstrap required, opened a new log
I20260807 06:38:43.801033 22914 ts_tablet_manager.cc:1397] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.801009 22919 raft_consensus.cc:348] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.801074 22919 raft_consensus.cc:374] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.801097 22919 raft_consensus.cc:729] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead773b5185742fcaf4e12ddc271134b, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.801153 22919 consensus_queue.cc:260] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.801196 22914 raft_consensus.cc:348] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.801255 22919 ts_tablet_manager.cc:1428] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.801262 22914 raft_consensus.cc:374] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.801282 22914 raft_consensus.cc:729] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635519f6fb3c4633bb02bd324aec8a73, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.801324 22914 consensus_queue.cc:260] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.801406 22914 ts_tablet_manager.cc:1428] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:38:43.862268 22920 raft_consensus.cc:482] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:43.862393 22920 raft_consensus.cc:504] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.862778 22920 leader_election.cc:288] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229), 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709)
I20260807 06:38:43.866896 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.863158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:54165 (local address 127.11.114.194:46709)
0807 06:38:43.863205 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.863207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.863301 (+ 94us) server_negotiation.cc:407] Connection header received
0807 06:38:43.863402 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.863403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.863413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.863444 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.863643 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.864151 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.866150 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.866499 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.866592 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.866604 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.866793 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.866827 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:43.866909 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.862962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:46709 (local address 127.11.114.195:54165)
0807 06:38:43.863238 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.863248 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.863327 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.863506 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.863506 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.863508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.863560 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.863561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.864234 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.864236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.866061 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.866063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.866605 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.866607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.866695 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.866699 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.866784 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.866826 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":82,"threads_started":1}
I20260807 06:38:43.866976 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.863016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:34939 (local address 127.11.114.193:43229)
0807 06:38:43.863065 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.863067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.863190 (+ 123us) server_negotiation.cc:407] Connection header received
0807 06:38:43.863323 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.863324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.863336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.863398 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:38:43.863599 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.864380 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.866000 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.866581 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.866655 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.866664 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.866867 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.866904 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:38:43.867197 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.862945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43229 (local address 127.11.114.195:34939)
0807 06:38:43.863128 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.863140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.863220 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.863436 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.863437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.863440 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:38:43.863522 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.863524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.864487 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.864488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.865905 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.865908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.866701 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.866702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.866790 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.866794 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.867062 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.867103 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260807 06:38:43.867406 22626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "822af26209794cf9b76fe718eef6ed1a" candidate_uuid: "635519f6fb3c4633bb02bd324aec8a73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" is_pre_election: true
I20260807 06:38:43.867494 22626 raft_consensus.cc:2454] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 635519f6fb3c4633bb02bd324aec8a73 in term 0.
I20260807 06:38:43.867450 22432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "822af26209794cf9b76fe718eef6ed1a" candidate_uuid: "635519f6fb3c4633bb02bd324aec8a73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead773b5185742fcaf4e12ddc271134b" is_pre_election: true
I20260807 06:38:43.867537 22432 raft_consensus.cc:2454] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 635519f6fb3c4633bb02bd324aec8a73 in term 0.
I20260807 06:38:43.867689 22728 leader_election.cc:302] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003fafab1d3a4d2086aaa0c8423dc5f1, 635519f6fb3c4633bb02bd324aec8a73; no voters:
I20260807 06:38:43.867847 22920 raft_consensus.cc:2781] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:43.867890 22920 raft_consensus.cc:482] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:43.867909 22920 raft_consensus.cc:3037] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.868609 22920 raft_consensus.cc:504] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.868755 22920 leader_election.cc:288] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [CANDIDATE]: Term 1 election: Requested vote from peers ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229), 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709)
I20260807 06:38:43.868929 22432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "822af26209794cf9b76fe718eef6ed1a" candidate_uuid: "635519f6fb3c4633bb02bd324aec8a73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead773b5185742fcaf4e12ddc271134b"
I20260807 06:38:43.869026 22432 raft_consensus.cc:3037] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.869693 22432 raft_consensus.cc:2454] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 635519f6fb3c4633bb02bd324aec8a73 in term 1.
I20260807 06:38:43.869818 22626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "822af26209794cf9b76fe718eef6ed1a" candidate_uuid: "635519f6fb3c4633bb02bd324aec8a73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1"
I20260807 06:38:43.869880 22725 leader_election.cc:302] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 635519f6fb3c4633bb02bd324aec8a73, ead773b5185742fcaf4e12ddc271134b; no voters:
I20260807 06:38:43.869902 22626 raft_consensus.cc:3037] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.869953 22920 raft_consensus.cc:2781] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:43.870076 22920 raft_consensus.cc:686] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 1 LEADER]: Becoming Leader. State: Replica: 635519f6fb3c4633bb02bd324aec8a73, State: Running, Role: LEADER
I20260807 06:38:43.870138 22920 consensus_queue.cc:237] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.870671 22626 raft_consensus.cc:2454] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 635519f6fb3c4633bb02bd324aec8a73 in term 1.
I20260807 06:38:43.871186 22231 catalog_manager.cc:5495] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195). New cstate: current_term: 1 leader_uuid: "635519f6fb3c4633bb02bd324aec8a73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } health_report { overall_health: HEALTHY } } }
I20260807 06:38:43.894451 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.891584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.11.114.195:40851)
0807 06:38:43.891639 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.891642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.891658 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:38:43.891789 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.891790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.891818 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.891898 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:43.892036 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.892816 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.893606 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.893912 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.893983 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.894109 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.894253 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.894327 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.894350 (+ 23us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.894371 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:38:43.895505 22626 raft_consensus.cc:1261] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Refusing update from remote peer 635519f6fb3c4633bb02bd324aec8a73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.895555 22432 raft_consensus.cc:1261] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Refusing update from remote peer 635519f6fb3c4633bb02bd324aec8a73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.895784 22920 consensus_queue.cc:1035] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.895942 23027 consensus_queue.cc:1035] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.899168 22230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:43.899256 22230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:43.899914 22230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: Sending DeleteTablet for 3 replicas of tablet 822af26209794cf9b76fe718eef6ed1a
I20260807 06:38:43.900151 22408 tablet_service.cc:1514] Processing DeleteTablet for tablet 822af26209794cf9b76fe718eef6ed1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:35462
I20260807 06:38:43.900229 22595 tablet_service.cc:1514] Processing DeleteTablet for tablet 822af26209794cf9b76fe718eef6ed1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:50360
I20260807 06:38:43.900259 22954 tablet_replica.cc:330] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: stopping tablet replica
I20260807 06:38:43.900319 22954 raft_consensus.cc:2229] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:43.900316 22792 tablet_service.cc:1514] Processing DeleteTablet for tablet 822af26209794cf9b76fe718eef6ed1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:51810
I20260807 06:38:43.900352 22954 raft_consensus.cc:2258] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.900393 22956 tablet_replica.cc:330] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: stopping tablet replica
I20260807 06:38:43.900441 22956 raft_consensus.cc:2229] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:43.900481 22956 raft_consensus.cc:2258] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.900552 22955 tablet_replica.cc:330] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: stopping tablet replica
I20260807 06:38:43.900596 22955 raft_consensus.cc:2229] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:43.900691 22955 raft_consensus.cc:2258] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.901510 22956 ts_tablet_manager.cc:1905] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.903103 22954 ts_tablet_manager.cc:1905] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.904529 22954 ts_tablet_manager.cc:1918] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:43.904589 22954 log.cc:1196] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/822af26209794cf9b76fe718eef6ed1a
I20260807 06:38:43.904806 22954 ts_tablet_manager.cc:1939] T 822af26209794cf9b76fe718eef6ed1a P ead773b5185742fcaf4e12ddc271134b: Deleting consensus metadata
I20260807 06:38:43.905198 22955 ts_tablet_manager.cc:1905] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.905397 22956 ts_tablet_manager.cc:1918] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:43.905449 22956 log.cc:1196] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/822af26209794cf9b76fe718eef6ed1a
I20260807 06:38:43.905699 22956 ts_tablet_manager.cc:1939] T 822af26209794cf9b76fe718eef6ed1a P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting consensus metadata
I20260807 06:38:43.906147 22230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:38:43.906322 22230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:38:43.906733 22955 ts_tablet_manager.cc:1918] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:38:43.906795 22955 log.cc:1196] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/822af26209794cf9b76fe718eef6ed1a
I20260807 06:38:43.907001 22955 ts_tablet_manager.cc:1939] T 822af26209794cf9b76fe718eef6ed1a P 635519f6fb3c4633bb02bd324aec8a73: Deleting consensus metadata
I20260807 06:38:43.907141 22209 catalog_manager.cc:4841] TS 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709): tablet 822af26209794cf9b76fe718eef6ed1a (table test-table [id=4ae72b5dbf134d318a823913734dfe92]) successfully deleted
I20260807 06:38:43.907469 22208 catalog_manager.cc:4841] TS ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229): tablet 822af26209794cf9b76fe718eef6ed1a (table test-table [id=4ae72b5dbf134d318a823913734dfe92]) successfully deleted
I20260807 06:38:43.907773 22209 catalog_manager.cc:4841] TS 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851): tablet 822af26209794cf9b76fe718eef6ed1a (table test-table [id=4ae72b5dbf134d318a823913734dfe92]) successfully deleted
I20260807 06:38:43.908386 22408 tablet_service.cc:1467] Processing CreateTablet for tablet 387b0f41f4f040ce8cb5f979392a3a85 (DEFAULT_TABLE table=test-table [id=c8adc9f2fe7d45f28de10c2aa7e48f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.908424 22792 tablet_service.cc:1467] Processing CreateTablet for tablet 387b0f41f4f040ce8cb5f979392a3a85 (DEFAULT_TABLE table=test-table [id=c8adc9f2fe7d45f28de10c2aa7e48f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.908504 22408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387b0f41f4f040ce8cb5f979392a3a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.908528 22792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387b0f41f4f040ce8cb5f979392a3a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.909617 22914 tablet_bootstrap.cc:492] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: Bootstrap starting.
I20260807 06:38:43.909758 22919 tablet_bootstrap.cc:492] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: Bootstrap starting.
I20260807 06:38:43.910049 22914 tablet_bootstrap.cc:654] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.910086 22919 tablet_bootstrap.cc:654] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.910458 22595 tablet_service.cc:1467] Processing CreateTablet for tablet 387b0f41f4f040ce8cb5f979392a3a85 (DEFAULT_TABLE table=test-table [id=c8adc9f2fe7d45f28de10c2aa7e48f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:38:43.910558 22595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387b0f41f4f040ce8cb5f979392a3a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:43.911307 22914 tablet_bootstrap.cc:492] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: No bootstrap required, opened a new log
I20260807 06:38:43.911358 22914 ts_tablet_manager.cc:1397] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:38:43.911512 22914 raft_consensus.cc:348] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.911590 22914 raft_consensus.cc:374] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.911610 22914 raft_consensus.cc:729] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635519f6fb3c4633bb02bd324aec8a73, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.911638 22914 consensus_queue.cc:260] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.911715 22914 ts_tablet_manager.cc:1428] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.911778 22917 tablet_bootstrap.cc:492] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: Bootstrap starting.
I20260807 06:38:43.912087 22917 tablet_bootstrap.cc:654] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:43.913020 22917 tablet_bootstrap.cc:492] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: No bootstrap required, opened a new log
I20260807 06:38:43.913069 22917 ts_tablet_manager.cc:1397] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:43.913151 22919 tablet_bootstrap.cc:492] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: No bootstrap required, opened a new log
I20260807 06:38:43.913194 22919 ts_tablet_manager.cc:1397] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:38:43.913175 22917 raft_consensus.cc:348] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.913236 22917 raft_consensus.cc:374] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.913257 22917 raft_consensus.cc:729] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003fafab1d3a4d2086aaa0c8423dc5f1, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.913295 22917 consensus_queue.cc:260] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.913290 22919 raft_consensus.cc:348] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.913349 22919 raft_consensus.cc:374] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:43.913369 22919 raft_consensus.cc:729] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead773b5185742fcaf4e12ddc271134b, State: Initialized, Role: FOLLOWER
I20260807 06:38:43.913373 22917 ts_tablet_manager.cc:1428] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.913407 22919 consensus_queue.cc:260] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.913491 22919 ts_tablet_manager.cc:1428] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:43.942873 22931 raft_consensus.cc:482] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:38:43.942988 22931 raft_consensus.cc:504] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.943166 22931 leader_election.cc:288] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709), 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851)
I20260807 06:38:43.943379 22626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387b0f41f4f040ce8cb5f979392a3a85" candidate_uuid: "ead773b5185742fcaf4e12ddc271134b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" is_pre_election: true
I20260807 06:38:43.943404 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387b0f41f4f040ce8cb5f979392a3a85" candidate_uuid: "ead773b5185742fcaf4e12ddc271134b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635519f6fb3c4633bb02bd324aec8a73" is_pre_election: true
I20260807 06:38:43.943481 22626 raft_consensus.cc:2454] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead773b5185742fcaf4e12ddc271134b in term 0.
I20260807 06:38:43.943495 22817 raft_consensus.cc:2454] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead773b5185742fcaf4e12ddc271134b in term 0.
I20260807 06:38:43.943620 22346 leader_election.cc:302] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 635519f6fb3c4633bb02bd324aec8a73, ead773b5185742fcaf4e12ddc271134b; no voters:
I20260807 06:38:43.943744 22931 raft_consensus.cc:2781] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:38:43.943797 22931 raft_consensus.cc:482] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:38:43.943819 22931 raft_consensus.cc:3037] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.944777 22931 raft_consensus.cc:504] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.944911 22931 leader_election.cc:288] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [CANDIDATE]: Term 1 election: Requested vote from peers 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709), 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851)
I20260807 06:38:43.945144 22626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387b0f41f4f040ce8cb5f979392a3a85" candidate_uuid: "ead773b5185742fcaf4e12ddc271134b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1"
I20260807 06:38:43.945175 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387b0f41f4f040ce8cb5f979392a3a85" candidate_uuid: "ead773b5185742fcaf4e12ddc271134b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635519f6fb3c4633bb02bd324aec8a73"
I20260807 06:38:43.945220 22626 raft_consensus.cc:3037] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.945245 22817 raft_consensus.cc:3037] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:43.946074 22817 raft_consensus.cc:2454] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead773b5185742fcaf4e12ddc271134b in term 1.
I20260807 06:38:43.946180 22626 raft_consensus.cc:2454] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead773b5185742fcaf4e12ddc271134b in term 1.
I20260807 06:38:43.946229 22346 leader_election.cc:302] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 635519f6fb3c4633bb02bd324aec8a73, ead773b5185742fcaf4e12ddc271134b; no voters:
I20260807 06:38:43.946410 22931 raft_consensus.cc:2781] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:43.946506 22931 raft_consensus.cc:686] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 1 LEADER]: Becoming Leader. State: Replica: ead773b5185742fcaf4e12ddc271134b, State: Running, Role: LEADER
I20260807 06:38:43.946549 22931 consensus_queue.cc:237] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } }
I20260807 06:38:43.947047 22230 catalog_manager.cc:5495] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b reported cstate change: term changed from 0 to 1, leader changed from <none> to ead773b5185742fcaf4e12ddc271134b (127.11.114.193). New cstate: current_term: 1 leader_uuid: "ead773b5185742fcaf4e12ddc271134b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead773b5185742fcaf4e12ddc271134b" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260807 06:38:43.963416 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.960359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.11.114.193:43229)
0807 06:38:43.960398 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.960400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.960409 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:43.960465 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.960466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.960476 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.960533 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:43.960662 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.961404 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.962562 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.962854 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.962929 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.963139 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.963176 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.963230 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.963320 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.963342 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:38:43.969107 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.966232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:34541 (local address 127.11.114.194:46709)
0807 06:38:43.966272 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.966274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.966283 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:43.966368 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.966369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.966380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.966438 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:43.966541 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.967241 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.968316 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.968600 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.968672 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.968860 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.968893 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.968944 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.969012 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.969034 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:38:43.969144 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.966021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:46709 (local address 127.11.114.193:34541)
0807 06:38:43.966202 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.966216 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.966271 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.966441 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.966442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.966444 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:43.966486 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.966487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.967332 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.967333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.968218 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.968220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.968685 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.968686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.968755 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.968771 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.968960 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.968962 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.969011 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.969060 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260807 06:38:43.972062 22817 raft_consensus.cc:1261] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Refusing update from remote peer ead773b5185742fcaf4e12ddc271134b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.972226 22626 raft_consensus.cc:1261] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Refusing update from remote peer ead773b5185742fcaf4e12ddc271134b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:38:43.972466 23037 consensus_queue.cc:1035] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [LEADER]: Connected to new peer: Peer: permanent_uuid: "003fafab1d3a4d2086aaa0c8423dc5f1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 46709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.973510 23037 consensus_queue.cc:1035] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [LEADER]: Connected to new peer: Peer: permanent_uuid: "635519f6fb3c4633bb02bd324aec8a73" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:38:43.982590 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.979621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:52185 (local address 127.11.114.193:43229)
0807 06:38:43.979691 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:43.979693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:43.979703 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:43.979724 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:43.979725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:43.979743 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:43.979802 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:38:43.979983 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.980680 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.981823 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.982099 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.982174 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.982325 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.982359 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.982418 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:43.982491 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 06:38:43.982515 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:38:43.982617 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:43.979366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43229 (local address 127.11.114.194:52185)
0807 06:38:43.979452 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:43.979464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:43.979503 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:43.979854 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:43.979855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:43.979856 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:38:43.979899 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.979900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.980785 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.980786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.981696 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:43.981699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:43.982170 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:43.982171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:43.982233 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:43.982241 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:38:43.982435 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:38:43.982438 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:38:43.982489 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:38:43.982542 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 06:38:43.993781 22230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:43.993911 22230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:38:43.994592 22230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f304f17c43cc44cfb846fe9ac0234865: Sending DeleteTablet for 3 replicas of tablet 387b0f41f4f040ce8cb5f979392a3a85
I20260807 06:38:43.994906 22408 tablet_service.cc:1514] Processing DeleteTablet for tablet 387b0f41f4f040ce8cb5f979392a3a85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:35462
I20260807 06:38:43.994997 22954 tablet_replica.cc:330] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: stopping tablet replica
I20260807 06:38:43.995050 22954 raft_consensus.cc:2229] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:38:43.995112 22954 raft_consensus.cc:2258] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.995635 22954 ts_tablet_manager.cc:1905] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:43.997259 22954 ts_tablet_manager.cc:1918] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:43.997311 22954 log.cc:1196] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/387b0f41f4f040ce8cb5f979392a3a85
I20260807 06:38:43.997572 22954 ts_tablet_manager.cc:1939] T 387b0f41f4f040ce8cb5f979392a3a85 P ead773b5185742fcaf4e12ddc271134b: Deleting consensus metadata
I20260807 06:38:43.997937 22208 catalog_manager.cc:4841] TS ead773b5185742fcaf4e12ddc271134b (127.11.114.193:43229): tablet 387b0f41f4f040ce8cb5f979392a3a85 (table test-table [id=c8adc9f2fe7d45f28de10c2aa7e48f7a]) successfully deleted
I20260807 06:38:43.998041 22595 tablet_service.cc:1514] Processing DeleteTablet for tablet 387b0f41f4f040ce8cb5f979392a3a85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:50360
I20260807 06:38:43.998143 22956 tablet_replica.cc:330] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: stopping tablet replica
I20260807 06:38:43.998201 22956 raft_consensus.cc:2229] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:43.998245 22956 raft_consensus.cc:2258] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.998541 22792 tablet_service.cc:1514] Processing DeleteTablet for tablet 387b0f41f4f040ce8cb5f979392a3a85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:38:43 UTC) from {username='oryx', principal='oryx/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:51810
I20260807 06:38:43.998636 22955 tablet_replica.cc:330] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: stopping tablet replica
I20260807 06:38:43.998679 22955 raft_consensus.cc:2229] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:38:43.998711 22955 raft_consensus.cc:2258] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:38:43.998835 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 22326
I20260807 06:38:44.006209 22956 ts_tablet_manager.cc:1905] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:44.009987 22956 ts_tablet_manager.cc:1918] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:44.010092 22956 log.cc:1196] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/387b0f41f4f040ce8cb5f979392a3a85
I20260807 06:38:44.010346 22956 ts_tablet_manager.cc:1939] T 387b0f41f4f040ce8cb5f979392a3a85 P 003fafab1d3a4d2086aaa0c8423dc5f1: Deleting consensus metadata
I20260807 06:38:44.010790 22209 catalog_manager.cc:4841] TS 003fafab1d3a4d2086aaa0c8423dc5f1 (127.11.114.194:46709): tablet 387b0f41f4f040ce8cb5f979392a3a85 (table test-table [id=c8adc9f2fe7d45f28de10c2aa7e48f7a]) successfully deleted
I20260807 06:38:44.010964 22955 ts_tablet_manager.cc:1905] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:38:44.012771 22955 ts_tablet_manager.cc:1918] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:38:44.012842 22955 log.cc:1196] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/387b0f41f4f040ce8cb5f979392a3a85
I20260807 06:38:44.013064 22955 ts_tablet_manager.cc:1939] T 387b0f41f4f040ce8cb5f979392a3a85 P 635519f6fb3c4633bb02bd324aec8a73: Deleting consensus metadata
I20260807 06:38:44.013489 22209 catalog_manager.cc:4841] TS 635519f6fb3c4633bb02bd324aec8a73 (127.11.114.195:40851): tablet 387b0f41f4f040ce8cb5f979392a3a85 (table test-table [id=c8adc9f2fe7d45f28de10c2aa7e48f7a]) successfully deleted
I20260807 06:38:44.016901 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 22503
I20260807 06:38:44.032210 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 22705
I20260807 06:38:44.045450 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 22186
2026-08-07T06:38:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2913 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:44 dist-test-slave-gkw7 krb5kdc[23056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:44 dist-test-slave-gkw7 krb5kdc[23056](info): set up 2 sockets
Aug 07 06:38:44 dist-test-slave-gkw7 krb5kdc[23056](info): commencing operation
krb5kdc: starting...
W20260807 06:38:45.260690 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:45 dist-test-slave-gkw7 krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:45.393399 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:42161
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:42161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:45.479502 23071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:45.479769 23071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:45.479810 23071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:45.480968 23071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:45.481035 23071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:45.481058 23071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:45.481079 23071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:45.481107 23071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:45.482682 23071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:42161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:42161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:45.482966 23071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:45.483201 23071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:45.488705 23087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:45.488664 23077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:45.488649 23081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:45.488801 23071 server_base.cc:1034] running on GCE node
I20260807 06:38:45.489172 23071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:45.489482 23071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:45.490731 23071 hybrid_clock.cc:648] HybridClock initialized: now 1786084725490716 us; error 34 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T06:38:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1451 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:38:45 dist-test-slave-gkw7 krb5kdc[23092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:38:45 dist-test-slave-gkw7 krb5kdc[23092](info): set up 2 sockets
Aug 07 06:38:45 dist-test-slave-gkw7 krb5kdc[23092](info): commencing operation
krb5kdc: starting...
W20260807 06:38:46.751808 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:38:46 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:38:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:38:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:38:46.876411 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:43979
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:43979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:46.959438 23113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:46.959651 23113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:46.959681 23113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:46.960924 23113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:46.960965 23113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:46.960980 23113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:46.961020 23113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:46.961040 23113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:46.962565 23113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:43979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:43979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:46.962826 23113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:46.963052 23113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:46.967964 23125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:46.967948 23121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:46.967934 23120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:46.968080 23113 server_base.cc:1034] running on GCE node
I20260807 06:38:46.968488 23113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:46.968786 23113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:46.969923 23113 hybrid_clock.cc:648] HybridClock initialized: now 1786084726969912 us; error 29 us; skew 500 ppm
Aug 07 06:38:46 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084726, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:46.980904 23113 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:46.981519 23113 webserver.cc:460] Webserver started at http://127.11.114.254:46621/ using document root <none> and password file <none>
I20260807 06:38:46.981869 23113 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:46.981923 23113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:46.982036 23113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:46.983239 23113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "4ab4f31197da40558f499a79ea3ec351"
format_stamp: "Formatted at 2026-08-07 06:38:46 on dist-test-slave-gkw7"
server_key: "3f3eaf18287e5bfbc50e80c2f1542b6d"
server_key_iv: "c465d9eea067d718163191224245b457"
server_key_version: "encryptionkey@0"
I20260807 06:38:46.983582 23113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "4ab4f31197da40558f499a79ea3ec351"
format_stamp: "Formatted at 2026-08-07 06:38:46 on dist-test-slave-gkw7"
server_key: "3f3eaf18287e5bfbc50e80c2f1542b6d"
server_key_iv: "c465d9eea067d718163191224245b457"
server_key_version: "encryptionkey@0"
I20260807 06:38:46.985033 23113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:46.985884 23135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:46.986063 23113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:46.986219 23113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "4ab4f31197da40558f499a79ea3ec351"
format_stamp: "Formatted at 2026-08-07 06:38:46 on dist-test-slave-gkw7"
server_key: "3f3eaf18287e5bfbc50e80c2f1542b6d"
server_key_iv: "c465d9eea067d718163191224245b457"
server_key_version: "encryptionkey@0"
I20260807 06:38:46.986305 23113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:46.997474 23113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:46.999568 23113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:46.999697 23113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:47.004256 23113 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:43979
I20260807 06:38:47.004289 23201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:43979 every 8 connection(s)
I20260807 06:38:47.004703 23113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:47.005685 23202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:38:47.008057 23202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Bootstrap starting.
I20260807 06:38:47.008680 23202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Neither blocks nor log segments found. Creating new log.
I20260807 06:38:47.008976 23202 log.cc:824] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:47.009784 23202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: No bootstrap required, opened a new log
I20260807 06:38:47.011389 23202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } }
I20260807 06:38:47.011552 23202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:38:47.011615 23202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab4f31197da40558f499a79ea3ec351, State: Initialized, Role: FOLLOWER
I20260807 06:38:47.011664 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 23113
I20260807 06:38:47.011786 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:38:47.011811 23202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } }
I20260807 06:38:47.011900 23202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:47.011937 23202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:47.011986 23202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:38:47.011996 11723 external_mini_cluster.cc:1428] Setting key 15148532025471d1ef24aae8db7e0147
I20260807 06:38:47.012581 23202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } }
I20260807 06:38:47.012725 23202 leader_election.cc:302] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ab4f31197da40558f499a79ea3ec351; no voters:
I20260807 06:38:47.012903 23202 leader_election.cc:288] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:38:47.012929 23213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:38:47.013034 23213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 1 LEADER]: Becoming Leader. State: Replica: 4ab4f31197da40558f499a79ea3ec351, State: Running, Role: LEADER
I20260807 06:38:47.013167 23202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:47.013172 23213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } }
I20260807 06:38:47.013765 23213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ab4f31197da40558f499a79ea3ec351. Latest consensus state: current_term: 1 leader_uuid: "4ab4f31197da40558f499a79ea3ec351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } } }
I20260807 06:38:47.013775 23215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ab4f31197da40558f499a79ea3ec351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } } }
I20260807 06:38:47.013860 23213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:47.013895 23215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: This master's current role is: LEADER
W20260807 06:38:47.015305 23233 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:38:47.015409 23233 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:38:47.015498 23220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:47.015671 23220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:47.017155 23220 catalog_manager.cc:1347] Generated new cluster ID: ecf1bcd249db4d6c98d58b676bcef7f2
I20260807 06:38:47.017216 23220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:47.024523 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.012500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53540 (local address 127.11.114.254:43979)
0807 06:38:47.012692 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:47.012703 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:47.012741 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:38:47.012888 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:47.012893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:47.012938 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:47.013087 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:47.013984 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.015310 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.016233 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.016769 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.016888 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.019755 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:47.019760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:47.019760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:47.019771 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:47.022535 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:47.022952 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:47.022955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:47.022956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:47.023066 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:47.023252 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:47.023253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:47.023253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:47.023549 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:47.023677 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:47.024153 (+ 476us) server_negotiation.cc:299] Negotiation successful
0807 06:38:47.024291 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260807 06:38:47.027829 23220 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:38:47.028539 23220 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:47.043888 23220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Generated new TSK 0
I20260807 06:38:47.044160 23220 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:38:47.073009 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43979
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:47.158751 23241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:47.159021 23241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:47.159070 23241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:47.160392 23241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:47.160463 23241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:47.160513 23241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:47.162190 23241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:47.162496 23241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:47.162757 23241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:47.168255 23252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:47.168263 23248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:47.168263 23250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:47.168409 23241 server_base.cc:1034] running on GCE node
I20260807 06:38:47.168704 23241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:47.168916 23241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:47.170053 23241 hybrid_clock.cc:648] HybridClock initialized: now 1786084727170046 us; error 30 us; skew 500 ppm
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084727, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:47.172717 23241 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:38:47.173205 23241 webserver.cc:460] Webserver started at http://127.11.114.193:34979/ using document root <none> and password file <none>
I20260807 06:38:47.173476 23241 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:47.173568 23241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:47.173758 23241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:47.174903 23241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "e6a371163ce646e6a85aa44b61e33799"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "f3dab1ec067fc973a6c7e89513aa5017"
server_key_iv: "b4ec6183a540845a3942ad358830d504"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.175287 23241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "e6a371163ce646e6a85aa44b61e33799"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "f3dab1ec067fc973a6c7e89513aa5017"
server_key_iv: "b4ec6183a540845a3942ad358830d504"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.176748 23241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:38:47.177522 23261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.177738 23241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:47.177826 23241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "e6a371163ce646e6a85aa44b61e33799"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "f3dab1ec067fc973a6c7e89513aa5017"
server_key_iv: "b4ec6183a540845a3942ad358830d504"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.177881 23241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:47.186492 23241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:47.188738 23241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:47.188855 23241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:47.189035 23241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:47.189419 23241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:47.189466 23241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.189503 23241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:47.189548 23241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.195438 23241 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:45537
I20260807 06:38:47.195528 23431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:45537 every 8 connection(s)
I20260807 06:38:47.195832 23241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:47.198607 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 23241
I20260807 06:38:47.198695 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:38:47.198912 11723 external_mini_cluster.cc:1428] Setting key d9f09bc62c55e3598cedc2bf39807a3d
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084727, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:47.206135 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.196574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:48075 (local address 127.11.114.254:43979)
0807 06:38:47.196629 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:47.196631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:47.196809 (+ 178us) server_negotiation.cc:407] Connection header received
0807 06:38:47.197365 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:47.197366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:47.197380 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:47.197432 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:47.198438 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.199133 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.200313 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.200541 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.200622 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.202974 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:47.202976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:47.202976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:47.202985 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:47.204679 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:47.205066 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:47.205067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:47.205067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:47.205137 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:47.205379 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:47.205380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:47.205381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:47.205586 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:47.205669 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:47.205917 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 06:38:47.206025 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:38:47.206156 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.196529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:48075)
0807 06:38:47.196676 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:47.196722 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:47.197253 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:47.197502 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:47.197506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:47.198137 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:47.198333 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.198338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.199258 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.199260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.200222 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.200225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.200630 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.200631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.200777 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.201330 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:47.201338 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:47.202884 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:47.204763 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:47.204769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:47.204771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:47.204989 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:47.205214 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:47.205214 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:47.205215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:47.205317 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:47.205655 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:47.205659 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:47.205778 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 06:38:47.205865 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:38:47.205988 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260807 06:38:47.206692 23433 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43979
I20260807 06:38:47.206808 23433 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:47.207016 23433 heartbeater.cc:511] Master 127.11.114.254:43979 requested a full tablet report, sending...
I20260807 06:38:47.207567 23154 ts_manager.cc:194] Registered new tserver with Master: e6a371163ce646e6a85aa44b61e33799 (127.11.114.193:45537)
I20260807 06:38:47.208184 23154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:48075
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:38:47.228436 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43979
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:47.315725 23440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:47.315935 23440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:47.315958 23440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:47.317461 23440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:47.317515 23440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:47.317579 23440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:47.319007 23440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:47.319205 23440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:47.319386 23440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:47.324585 23449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:47.324595 23448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:47.324707 23440 server_base.cc:1034] running on GCE node
W20260807 06:38:47.324595 23451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:47.325060 23440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:47.325304 23440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:47.326472 23440 hybrid_clock.cc:648] HybridClock initialized: now 1786084727326466 us; error 36 us; skew 500 ppm
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084727, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:47.328984 23440 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:38:47.329478 23440 webserver.cc:460] Webserver started at http://127.11.114.194:41235/ using document root <none> and password file <none>
I20260807 06:38:47.329751 23440 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:47.329826 23440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:47.329973 23440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:47.331027 23440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "cbd83ddce26845718009d22471565360"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "948bbabd6d6fb1d7560720e3d3ec14c1"
server_key_iv: "ee98783140ec3ba9e9ff14cbfd950b3d"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.331404 23440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "cbd83ddce26845718009d22471565360"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "948bbabd6d6fb1d7560720e3d3ec14c1"
server_key_iv: "ee98783140ec3ba9e9ff14cbfd950b3d"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.332742 23440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:38:47.333505 23459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.333753 23440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:47.333873 23440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "cbd83ddce26845718009d22471565360"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "948bbabd6d6fb1d7560720e3d3ec14c1"
server_key_iv: "ee98783140ec3ba9e9ff14cbfd950b3d"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.333949 23440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:47.346197 23440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:47.348178 23440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:47.348340 23440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:47.348608 23440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:47.348968 23440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:47.349030 23440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.349066 23440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:47.349108 23440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.355033 23440 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:39953
I20260807 06:38:47.355168 23623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:39953 every 8 connection(s)
I20260807 06:38:47.355455 23440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084727, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:47.364368 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 23440
I20260807 06:38:47.364526 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:38:47.364740 11723 external_mini_cluster.cc:1428] Setting key bea1909747459bfd7c2d0ac9f9c63eeb
I20260807 06:38:47.365190 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.356253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:33557 (local address 127.11.114.254:43979)
0807 06:38:47.356319 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:47.356321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:47.356501 (+ 180us) server_negotiation.cc:407] Connection header received
0807 06:38:47.356984 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:47.356984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:47.356995 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:47.357056 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:47.358131 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.358656 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.359683 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.359877 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.359936 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.362128 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:47.362130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:47.362130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:47.362137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:47.363578 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:47.363960 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:47.363961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:47.363962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:47.364036 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:47.364378 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:47.364379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:47.364379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:47.364570 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:47.364645 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:47.364971 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 06:38:47.365057 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:38:47.365360 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.356202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:33557)
0807 06:38:47.356368 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:47.356415 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:47.356897 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:47.357115 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:47.357119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:47.357807 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:47.358013 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.358017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.358745 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.358746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.359600 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.359602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.359951 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.359952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.360055 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.360536 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:47.360558 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:47.362001 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:47.363652 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:47.363658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:47.363660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:47.363877 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:47.364156 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:47.364157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:47.364157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:47.364312 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:47.364667 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:47.364674 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:47.364831 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 06:38:47.364947 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 06:38:47.365117 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260807 06:38:47.365988 23624 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43979
I20260807 06:38:47.366098 23624 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:47.366309 23624 heartbeater.cc:511] Master 127.11.114.254:43979 requested a full tablet report, sending...
I20260807 06:38:47.366676 23154 ts_manager.cc:194] Registered new tserver with Master: cbd83ddce26845718009d22471565360 (127.11.114.194:39953)
I20260807 06:38:47.367062 23154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:33557
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:38:47.392556 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43979
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:47.479018 23630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:47.479247 23630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:47.479274 23630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:47.480391 23630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:47.480432 23630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:47.480460 23630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:47.482033 23630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:47.482247 23630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:47.482450 23630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:47.487846 23642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:47.487792 23637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:47.487778 23640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:47.487917 23630 server_base.cc:1034] running on GCE node
I20260807 06:38:47.488252 23630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:47.488497 23630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:47.489655 23630 hybrid_clock.cc:648] HybridClock initialized: now 1786084727489636 us; error 38 us; skew 500 ppm
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084727, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:47.492254 23630 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:38:47.492700 23630 webserver.cc:460] Webserver started at http://127.11.114.195:42017/ using document root <none> and password file <none>
I20260807 06:38:47.492966 23630 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:47.493034 23630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:47.493193 23630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:38:47.494343 23630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "d0a5eda12c354513bbe18c900b75e370"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "4330d939717f1f177dd34e2e6b15afb4"
server_key_iv: "732e3261b102477f53a6f1a3294b3ce6"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.494695 23630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "d0a5eda12c354513bbe18c900b75e370"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "4330d939717f1f177dd34e2e6b15afb4"
server_key_iv: "732e3261b102477f53a6f1a3294b3ce6"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.496088 23630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:38:47.496833 23651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.496994 23630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:47.497093 23630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "d0a5eda12c354513bbe18c900b75e370"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "4330d939717f1f177dd34e2e6b15afb4"
server_key_iv: "732e3261b102477f53a6f1a3294b3ce6"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.497164 23630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:47.509936 23630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:47.511816 23630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:47.511976 23630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:47.512198 23630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:47.512523 23630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:47.512574 23630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.512616 23630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:47.512651 23630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.518945 23630 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:46277
I20260807 06:38:47.519037 23815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:46277 every 8 connection(s)
I20260807 06:38:47.519346 23630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084727, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:38:47.528585 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 23630
I20260807 06:38:47.528698 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:38:47.528772 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.520155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:42875 (local address 127.11.114.254:43979)
0807 06:38:47.520200 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:47.520203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:47.520410 (+ 207us) server_negotiation.cc:407] Connection header received
0807 06:38:47.520897 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:47.520898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:47.520911 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:47.520984 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:47.522055 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.522592 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.523573 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.523763 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.523826 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.526006 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:47.526007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:47.526007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:47.526014 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:47.527394 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:47.527841 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:47.527841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:47.527842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:47.527896 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:47.528106 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:47.528107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:47.528107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:47.528210 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:47.528261 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:47.528562 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 06:38:47.528657 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:38:47.528911 11723 external_mini_cluster.cc:1428] Setting key 691af3135b55353d57f96404413f859e
I20260807 06:38:47.528942 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.520109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:42875)
0807 06:38:47.520284 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:47.520326 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:47.520800 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:47.521039 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:47.521042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:47.521733 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:47.521961 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.521967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.522711 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.522711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.523500 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.523502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.523839 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.523839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.523939 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.524448 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:47.524459 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:47.525918 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:38:47.527503 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:47.527509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:47.527511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:47.527770 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:47.527952 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:38:47.527953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:38:47.527953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:38:47.528052 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:38:47.528282 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:38:47.528287 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:38:47.528423 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 06:38:47.528511 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 06:38:47.528634 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260807 06:38:47.529402 23816 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43979
I20260807 06:38:47.529521 23816 heartbeater.cc:464] Registering TS with master...
I20260807 06:38:47.529778 23816 heartbeater.cc:511] Master 127.11.114.254:43979 requested a full tablet report, sending...
I20260807 06:38:47.530155 23152 ts_manager.cc:194] Registered new tserver with Master: d0a5eda12c354513bbe18c900b75e370 (127.11.114.195:46277)
I20260807 06:38:47.530537 23152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:42875
I20260807 06:38:47.540738 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab.
Entry for principal oryx/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab.
WARNING: no policy specified for oryx/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab.
Entry for principal oryx/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab.
WARNING: no policy specified for oryx/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab.
Entry for principal oryx/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab.
I20260807 06:38:47.616693 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 23241
I20260807 06:38:47.633809 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 23440
I20260807 06:38:47.640571 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 23630
I20260807 06:38:47.650308 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 23113
I20260807 06:38:47.663725 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:43979
--webserver_interface=127.11.114.254
--webserver_port=46621
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:43979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:38:47.745708 23826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:47.745914 23826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:47.745936 23826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:47.747082 23826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:38:47.747125 23826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:47.747139 23826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:47.747171 23826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:38:47.747184 23826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:38:47.748718 23826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:43979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:43979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=46621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:47.748970 23826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:47.749207 23826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:47.754056 23836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:47.754051 23834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:47.754048 23838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:47.754345 23826 server_base.cc:1034] running on GCE node
I20260807 06:38:47.754545 23826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:47.754774 23826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:47.755918 23826 hybrid_clock.cc:648] HybridClock initialized: now 1786084727755903 us; error 28 us; skew 500 ppm
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084727, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:47.758543 23826 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:38:47.758952 23826 webserver.cc:460] Webserver started at http://127.11.114.254:46621/ using document root <none> and password file <none>
I20260807 06:38:47.759168 23826 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:47.759222 23826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:47.760501 23826 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:47.761089 23848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.761291 23826 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:38:47.761394 23826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "4ab4f31197da40558f499a79ea3ec351"
format_stamp: "Formatted at 2026-08-07 06:38:46 on dist-test-slave-gkw7"
server_key: "3f3eaf18287e5bfbc50e80c2f1542b6d"
server_key_iv: "c465d9eea067d718163191224245b457"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.761790 23826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:47.777839 23826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:47.780190 23826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:47.780350 23826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:47.784348 23826 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:43979
I20260807 06:38:47.784394 23929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:43979 every 8 connection(s)
I20260807 06:38:47.784794 23826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:38:47.786355 23931 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:38:47.787041 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Bootstrap starting.
I20260807 06:38:47.789085 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 23826
I20260807 06:38:47.789464 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:45537
--local_ip_for_outbound_sockets=127.11.114.193
--tserver_master_addrs=127.11.114.254:43979
--webserver_port=34979
--webserver_interface=127.11.114.193
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.114.193
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:38:47.795821 23931 log.cc:824] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Log is configured to *not* fsync() on all Append() calls
I20260807 06:38:47.797627 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:38:47.797930 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Bootstrap complete.
I20260807 06:38:47.800196 23931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } }
I20260807 06:38:47.800420 23931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab4f31197da40558f499a79ea3ec351, State: Initialized, Role: FOLLOWER
I20260807 06:38:47.800585 23931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } }
I20260807 06:38:47.800655 23931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:38:47.800711 23931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:38:47.800767 23931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:38:47.801837 23931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } }
I20260807 06:38:47.801945 23931 leader_election.cc:302] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ab4f31197da40558f499a79ea3ec351; no voters:
I20260807 06:38:47.802142 23931 leader_election.cc:288] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:38:47.802177 23935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:38:47.802354 23935 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [term 2 LEADER]: Becoming Leader. State: Replica: 4ab4f31197da40558f499a79ea3ec351, State: Running, Role: LEADER
I20260807 06:38:47.802428 23931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:38:47.802505 23935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } }
I20260807 06:38:47.802845 23935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ab4f31197da40558f499a79ea3ec351. Latest consensus state: current_term: 2 leader_uuid: "4ab4f31197da40558f499a79ea3ec351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } } }
I20260807 06:38:47.802850 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4ab4f31197da40558f499a79ea3ec351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4f31197da40558f499a79ea3ec351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43979 } } }
I20260807 06:38:47.803020 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:47.803296 23935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351 [sys.catalog]: This master's current role is: LEADER
I20260807 06:38:47.803609 23952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:38:47.804174 23952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:38:47.804361 23952 catalog_manager.cc:1259] Loaded cluster ID: ecf1bcd249db4d6c98d58b676bcef7f2
I20260807 06:38:47.804489 23952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:38:47.805675 23952 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:38:47.805856 23952 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ab4f31197da40558f499a79ea3ec351: Loaded TSK: 0
I20260807 06:38:47.806073 23952 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:38:47.875885 23933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:47.876142 23933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:47.876186 23933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:47.877368 23933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:47.877439 23933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:47.877485 23933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:38:47.878990 23933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.193.keytab
--principal=oryx/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:45537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=34979
--tserver_master_addrs=127.11.114.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:47.879267 23933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:47.879510 23933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:47.885712 23967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:47.885922 23933 server_base.cc:1034] running on GCE node
W20260807 06:38:47.885759 23971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:47.885725 23968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:47.886266 23933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:47.886523 23933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:47.887705 23933 hybrid_clock.cc:648] HybridClock initialized: now 1786084727887680 us; error 41 us; skew 500 ppm
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084727, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:47.890568 23933 init.cc:377] Logged in from keytab as oryx/127.11.114.193@KRBTEST.COM (short username oryx)
I20260807 06:38:47.891059 23933 webserver.cc:460] Webserver started at http://127.11.114.193:34979/ using document root <none> and password file <none>
I20260807 06:38:47.891292 23933 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:47.891381 23933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:47.892846 23933 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:47.893517 23979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.893769 23933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:38:47.893882 23933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "e6a371163ce646e6a85aa44b61e33799"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "f3dab1ec067fc973a6c7e89513aa5017"
server_key_iv: "b4ec6183a540845a3942ad358830d504"
server_key_version: "encryptionkey@0"
I20260807 06:38:47.894251 23933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:47.909649 23933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:47.912042 23933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:47.912221 23933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:47.912483 23933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:47.912809 23933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:47.912864 23933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.912907 23933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:47.912942 23933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:47.919486 23933 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:45537
I20260807 06:38:47.919531 24143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:45537 every 8 connection(s)
I20260807 06:38:47.919945 23933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:38:47.925590 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 23933
I20260807 06:38:47.925944 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:39953
--local_ip_for_outbound_sockets=127.11.114.194
--tserver_master_addrs=127.11.114.254:43979
--webserver_port=41235
--webserver_interface=127.11.114.194
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.114.194
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:47.930655 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.920670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:58745)
0807 06:38:47.920879 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:47.920929 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:47.921571 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:47.922046 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:47.922050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:47.922808 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:47.923080 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.923086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.924195 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.924196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.925478 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.925482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.926461 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.926462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.926615 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.927285 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:47.927296 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:47.930423 (+ 3127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
W20260807 06:38:47.930789 24147 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:47.920784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:58745 (local address 127.11.114.254:43979)
0807 06:38:47.921657 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:47.921665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:47.921696 (+ 31us) server_negotiation.cc:407] Connection header received
0807 06:38:47.921816 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:47.921821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:47.921862 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:47.922004 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:47.923190 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.924087 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.925629 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.926346 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.926459 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.930586 (+ 4127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:58745: BlockingRecv error: recv got EOF from 127.11.114.193:58745 (error 108)
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":81,"threads_started":1}
W20260807 06:38:47.931176 24144 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:43979 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.114.254:43979: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:47.937269 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.931418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:54285)
0807 06:38:47.931472 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:47.931485 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:47.931529 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:47.931684 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:47.931685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:47.931866 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:47.931934 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.931936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.932865 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.932866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.933818 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.933822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.934296 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.934297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.934382 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.934894 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:47.934905 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:47.937201 (+ 2296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 06:38:47.937408 24147 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:47.931483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:54285 (local address 127.11.114.254:43979)
0807 06:38:47.931524 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:47.931526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:47.931536 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:47.931600 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:47.931601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:47.931614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:47.931679 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:47.932014 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.932779 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.933905 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.934214 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.934293 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.937324 (+ 3031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:54285: BlockingRecv error: recv got EOF from 127.11.114.193:54285 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:47 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:47.942809 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:47.937848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:44827)
0807 06:38:47.937894 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:47.937904 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:47.937937 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:47.938088 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:47.938089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:47.938226 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:47.938263 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.938265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.939232 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.939233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.939989 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:47.939992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.940456 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.940457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:47.940500 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.940989 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:47.940994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:47.942759 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:38:47.942917 24147 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:47.937918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:44827 (local address 127.11.114.254:43979)
0807 06:38:47.937957 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:47.937959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:47.937970 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:38:47.937995 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:47.937996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:47.938008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:47.938075 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:47.938341 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.939143 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.940080 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:47.940386 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:47.940452 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:47.942857 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:44827: BlockingRecv error: recv got EOF from 127.11.114.193:44827 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 06:38:47.943077 24144 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:38:48.014081 24150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:48.014336 24150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:48.014379 24150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:48.015561 24150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:48.015637 24150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:48.015684 24150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:38:48.017161 24150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.194.keytab
--principal=oryx/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:39953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=41235
--tserver_master_addrs=127.11.114.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:48.017441 24150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:48.017722 24150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:48.022881 24161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:48.022938 24150 server_base.cc:1034] running on GCE node
W20260807 06:38:48.022889 24160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:48.022897 24164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:48.023264 24150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:48.023533 24150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:48.024682 24150 hybrid_clock.cc:648] HybridClock initialized: now 1786084728024660 us; error 42 us; skew 500 ppm
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084728, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:48.027199 24150 init.cc:377] Logged in from keytab as oryx/127.11.114.194@KRBTEST.COM (short username oryx)
I20260807 06:38:48.027659 24150 webserver.cc:460] Webserver started at http://127.11.114.194:41235/ using document root <none> and password file <none>
I20260807 06:38:48.027854 24150 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:48.027926 24150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:48.029263 24150 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:38:48.029896 24173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:48.030062 24150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:48.030150 24150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "cbd83ddce26845718009d22471565360"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "948bbabd6d6fb1d7560720e3d3ec14c1"
server_key_iv: "ee98783140ec3ba9e9ff14cbfd950b3d"
server_key_version: "encryptionkey@0"
I20260807 06:38:48.030493 24150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:48.048578 24150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:48.050984 24150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:48.051112 24150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:48.051323 24150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:48.051594 24150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:48.051625 24150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:48.051647 24150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:48.051661 24150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:48.057664 24150 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:39953
I20260807 06:38:48.057757 24347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:39953 every 8 connection(s)
I20260807 06:38:48.058094 24150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:38:48.062614 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 24150
I20260807 06:38:48.062958 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:46277
--local_ip_for_outbound_sockets=127.11.114.195
--tserver_master_addrs=127.11.114.254:43979
--webserver_port=42017
--webserver_interface=127.11.114.195
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.114.195
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:38:48.066481 24147 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:48.058911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:58855 (local address 127.11.114.254:43979)
0807 06:38:48.058979 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:48.058981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:48.059203 (+ 222us) server_negotiation.cc:407] Connection header received
0807 06:38:48.059786 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:48.059787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:48.059804 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:48.059856 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:48.060838 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.061342 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.062413 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.062611 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.062695 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.066432 (+ 3737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:58855: BlockingRecv error: recv got EOF from 127.11.114.194:58855 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:38:48.066488 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:48.058897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:58855)
0807 06:38:48.059083 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:48.059120 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:48.059649 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:48.059929 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:48.059933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:48.060525 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:48.060726 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.060730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.061434 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.061436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.062320 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.062322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.062708 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.062709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.062857 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.063381 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:48.063390 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:48.066245 (+ 2855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
W20260807 06:38:48.066838 24348 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:43979 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.114.254:43979: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:48.071767 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:48.067258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:59947)
0807 06:38:48.067329 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:48.067340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:48.067396 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:48.067557 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:48.067559 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:48.067732 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:48.067789 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.067791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.068468 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.068469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.068994 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.068996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.069350 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.069351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.069417 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.069947 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:48.069951 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:48.071718 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 06:38:48.071928 24147 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:48.067257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:59947 (local address 127.11.114.254:43979)
0807 06:38:48.067324 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:48.067327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:48.067379 (+ 52us) server_negotiation.cc:407] Connection header received
0807 06:38:48.067458 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:48.067459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:48.067469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:48.067537 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:48.067866 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.068387 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.069084 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.069279 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.069353 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.071845 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:59947: BlockingRecv error: recv got EOF from 127.11.114.194:59947 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:48.077054 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:48.072231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:47341)
0807 06:38:48.072284 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:48.072295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:48.072334 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:48.072458 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:48.072459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:48.072615 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:48.072657 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.072660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.073342 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.073344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.074300 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.074303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.074671 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.074672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.074739 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.075172 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:48.075177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:48.076986 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:38:48.077198 24147 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:48.072282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:47341 (local address 127.11.114.254:43979)
0807 06:38:48.072317 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:48.072318 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:48.072331 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:38:48.072396 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:48.072396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:48.072404 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:48.072451 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:48.072756 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.073254 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.074396 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.074579 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.074656 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.077116 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:47341: BlockingRecv error: recv got EOF from 127.11.114.194:47341 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 06:38:48.077404 24348 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:38:48.149628 24355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:38:48.149879 24355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:38:48.149919 24355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:38:48.151075 24355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:38:48.151144 24355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:38:48.151190 24355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:38:48.152673 24355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/krb5kdc/oryx_127.11.114.195.keytab
--principal=oryx/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:46277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=42017
--tserver_master_addrs=127.11.114.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:38:48.152944 24355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:38:48.153173 24355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:38:48.158299 24363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:48.158416 24362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:38:48.158429 24366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:38:48.158604 24355 server_base.cc:1034] running on GCE node
I20260807 06:38:48.158802 24355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:38:48.159010 24355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:38:48.160230 24355 hybrid_clock.cc:648] HybridClock initialized: now 1786084728160215 us; error 24 us; skew 500 ppm
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084728, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:38:48.163020 24355 init.cc:377] Logged in from keytab as oryx/127.11.114.195@KRBTEST.COM (short username oryx)
I20260807 06:38:48.163506 24355 webserver.cc:460] Webserver started at http://127.11.114.195:42017/ using document root <none> and password file <none>
I20260807 06:38:48.163793 24355 fs_manager.cc:359] Metadata directory not provided
I20260807 06:38:48.163895 24355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:38:48.165436 24355 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:38:48.166107 24374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:48.166278 24355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:48.166384 24355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "d0a5eda12c354513bbe18c900b75e370"
format_stamp: "Formatted at 2026-08-07 06:38:47 on dist-test-slave-gkw7"
server_key: "4330d939717f1f177dd34e2e6b15afb4"
server_key_iv: "732e3261b102477f53a6f1a3294b3ce6"
server_key_version: "encryptionkey@0"
I20260807 06:38:48.166733 24355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:38:48.182957 24355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:38:48.185422 24355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:38:48.185616 24355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:38:48.185869 24355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:38:48.186194 24355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:38:48.186249 24355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:48.186291 24355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:38:48.186324 24355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:38:48.192353 24355 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:46277
I20260807 06:38:48.192436 24538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:46277 every 8 connection(s)
I20260807 06:38:48.192792 24355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:48.199913 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 24355
I20260807 06:38:48.200248 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:48.193515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:44843)
0807 06:38:48.193708 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:48.193745 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:48.194255 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:48.194462 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:48.194465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:48.195044 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:48.195266 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.195271 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.195899 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.195900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.196771 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.196774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.197101 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.197102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.197234 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.197852 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:48.197861 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:48.200043 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
W20260807 06:38:48.200225 24147 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:48.193571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:44843 (local address 127.11.114.254:43979)
0807 06:38:48.193616 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:48.193619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:48.193820 (+ 201us) server_negotiation.cc:407] Connection header received
0807 06:38:48.194340 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:48.194341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:48.194352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:48.194406 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:48.195343 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.195820 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.196844 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.197031 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.197084 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.200149 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:44843: BlockingRecv error: recv got EOF from 127.11.114.195:44843 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 06:38:48.200453 24539 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:43979 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.114.254:43979: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:48.206175 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:48.200900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:35199)
0807 06:38:48.200950 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:48.200960 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:48.200997 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:48.201226 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:48.201227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:48.201353 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:48.201413 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.201415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.202120 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.202120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.202697 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.202699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.203119 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.203120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.203204 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.203655 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:48.203661 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:48.206078 (+ 2417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 06:38:48.206239 24548 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:48.200968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:35199 (local address 127.11.114.254:43979)
0807 06:38:48.201105 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:48.201106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:48.201112 (+ 6us) server_negotiation.cc:407] Connection header received
0807 06:38:48.201129 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:48.201129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:48.201138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:48.201209 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:48.201492 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.202019 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.202777 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.203058 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.203106 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.206158 (+ 3052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:35199: BlockingRecv error: recv got EOF from 127.11.114.195:35199 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260807 06:38:48.208024 24147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:48.200616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.11.114.254:43979)
0807 06:38:48.200662 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:48.200664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:48.200673 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:48.200695 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:48.200696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:48.200706 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:48.200768 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:48.201440 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.201990 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.202966 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.203187 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.203259 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.204705 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:38:48.204709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:38:48.204710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:38:48.204721 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:38:48.206583 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:48.206989 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:48.206993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:48.206993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:48.207093 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:38:48.207319 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:38:48.207320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:38:48.207321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:38:48.207552 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:38:48.207693 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 06:38:48.207818 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 06:38:48.207933 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:48.210901 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:48.206701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:49025)
0807 06:38:48.206808 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:48.206819 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:48.206870 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:48.207003 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:48.207004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:48.207130 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:48.207161 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.207162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.207786 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.207787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.208525 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.208528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.208978 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.208979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.209036 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.209318 (+ 282us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:48.209321 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:48.210821 (+ 1500us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260807 06:38:48.210948 24548 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:48.206759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:49025 (local address 127.11.114.254:43979)
0807 06:38:48.206805 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:48.206806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:48.206854 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:38:48.206925 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:48.206926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:48.206934 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:48.206998 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:48.207245 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.207709 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.208608 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.208910 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.208970 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.210890 (+ 1920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:49025: BlockingRecv error: recv got EOF from 127.11.114.195:49025 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 06:38:48.211069 24539 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:48 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:48.948544 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:48.943445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:59667)
0807 06:38:48.943608 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:48.943622 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:48.943697 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:48.943855 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:48.943856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:48.944076 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:48.944135 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.944136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.945127 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.945128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.945726 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:48.945728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.946075 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.946077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:48.946129 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.946575 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:48.946580 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:48.948475 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
W20260807 06:38:48.948637 24550 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:48.943549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:59667 (local address 127.11.114.254:43979)
0807 06:38:48.943686 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:48.943688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:48.943697 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:48.943760 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:48.943760 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:48.943771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:48.943831 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:48.944202 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.944986 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.945798 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:48.945986 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:48.946038 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:48.948565 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:59667: BlockingRecv error: recv got EOF from 127.11.114.193:59667 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
Aug 07 06:38:49 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:49 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:49.082643 24551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:49.077814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:48903)
0807 06:38:49.077992 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:49.078004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:49.078090 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:49.078243 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:49.078244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:49.078434 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:49.078505 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:49.078507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.079169 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.079170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:49.079718 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:49.079720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.080088 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.080089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:49.080163 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:49.080636 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:49.080641 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:49.082540 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
W20260807 06:38:49.082700 24550 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:49.077910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:48903 (local address 127.11.114.254:43979)
0807 06:38:49.077984 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:49.077986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:49.078070 (+ 84us) server_negotiation.cc:407] Connection header received
0807 06:38:49.078157 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:49.078158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:49.078171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:49.078250 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:49.078585 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.079091 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.079815 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.080018 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.080092 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:49.082636 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:48903: BlockingRecv error: recv got EOF from 127.11.114.194:48903 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 06:38:49 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:49 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:49.216570 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:49.211605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:37333)
0807 06:38:49.211775 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:49.211786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:49.211858 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:49.212029 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:49.212030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:49.212216 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:49.212275 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:49.212277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.213170 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.213172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:49.213858 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:49.213861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.214179 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.214180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:49.214225 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:49.214654 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:49.214659 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:49.216500 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
W20260807 06:38:49.216639 24550 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:49.211675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:37333 (local address 127.11.114.254:43979)
0807 06:38:49.211785 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:49.211787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:49.211858 (+ 71us) server_negotiation.cc:407] Connection header received
0807 06:38:49.211908 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:49.211909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:49.211923 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:49.211979 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:49.212401 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.213054 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.213932 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.214114 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.214168 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:49.216588 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:37333: BlockingRecv error: recv got EOF from 127.11.114.195:37333 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 06:38:49 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:49 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:49.954186 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:49.949138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:55169)
0807 06:38:49.949310 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:49.949325 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:49.949398 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:49.949593 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:49.949594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:49.949763 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:49.949817 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:49.949818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.950481 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.950483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:49.951003 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:49.951004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.951596 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.951597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:49.951658 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:49.952202 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:49.952207 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:49.954127 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
W20260807 06:38:49.954293 24555 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:49.949214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:55169 (local address 127.11.114.254:43979)
0807 06:38:49.949386 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:49.949388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:49.949398 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:49.949448 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:49.949448 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:49.949459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:49.949521 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:49.949883 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.950392 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.951134 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:49.951475 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:49.951559 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:49.954220 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:55169: BlockingRecv error: recv got EOF from 127.11.114.193:55169 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
Aug 07 06:38:50 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:50 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:50.087813 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:50.083175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:51255)
0807 06:38:50.083322 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:50.083334 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:50.083401 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:50.083548 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:50.083549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:50.083715 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:50.083768 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:50.083770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.084481 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.084482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:50.084995 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:50.084996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.085398 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.085399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:50.085439 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:50.085944 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:50.085949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:50.087760 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
W20260807 06:38:50.087984 24555 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:50.083271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:51255 (local address 127.11.114.254:43979)
0807 06:38:50.083335 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:50.083337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:50.083409 (+ 72us) server_negotiation.cc:407] Connection header received
0807 06:38:50.083445 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:50.083446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:50.083459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:50.083538 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:50.083850 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.084405 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.085126 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.085324 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.085391 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:50.087893 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:51255: BlockingRecv error: recv got EOF from 127.11.114.194:51255 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 06:38:50 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:50 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:50.221863 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:50.217177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:54979)
0807 06:38:50.217372 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:50.217384 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:50.217445 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:50.217590 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:50.217590 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:50.217771 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:50.217824 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:50.217825 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.218463 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.218463 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:50.218987 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:50.218989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.219301 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.219302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:50.219344 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:50.219777 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:50.219781 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:50.221762 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":143,"threads_started":1}
W20260807 06:38:50.221915 24555 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:50.217312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:54979 (local address 127.11.114.254:43979)
0807 06:38:50.217393 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:50.217395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:50.217443 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:38:50.217506 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:50.217507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:50.217521 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:50.217592 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:50.217889 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.218396 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.219055 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.219242 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.219293 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:50.221867 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:54979: BlockingRecv error: recv got EOF from 127.11.114.195:54979 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 06:38:50 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:50 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:50.959515 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:50.954742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:43565)
0807 06:38:50.954891 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:50.954905 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:50.954982 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:50.955229 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:50.955231 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:50.955424 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:50.955474 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:50.955476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.956166 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.956167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:50.956712 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:50.956713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.957080 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.957081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:50.957136 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:50.957677 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:50.957682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:50.959458 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
W20260807 06:38:50.959717 24559 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:50.954841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:43565 (local address 127.11.114.254:43979)
0807 06:38:50.955017 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:50.955023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:50.955035 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:38:50.955068 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:50.955068 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:50.955080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:50.955217 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:50.955554 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.956073 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.956796 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:50.957007 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:50.957083 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:50.959609 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:43565: BlockingRecv error: recv got EOF from 127.11.114.193:43565 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
Aug 07 06:38:51 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:51 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:51.093290 24560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:51.088341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:54271)
0807 06:38:51.088535 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:51.088552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:51.088641 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:51.088806 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:51.088807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:51.089000 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:51.089053 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:51.089055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.089849 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.089853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:51.090431 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:51.090433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.090799 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.090800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:51.090842 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:51.091310 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:51.091314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:51.093219 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
W20260807 06:38:51.093441 24559 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:51.088463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:54271 (local address 127.11.114.254:43979)
0807 06:38:51.088529 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:51.088531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:51.088620 (+ 89us) server_negotiation.cc:407] Connection header received
0807 06:38:51.088691 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:51.088692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:51.088705 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:51.088759 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:51.089154 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.089769 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.090515 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.090734 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.090794 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:51.093334 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:54271: BlockingRecv error: recv got EOF from 127.11.114.194:54271 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 07 06:38:51 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:51 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:51.227304 24561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:51.222324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:51833)
0807 06:38:51.222452 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:51.222465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:51.222531 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:51.222688 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:51.222689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:51.222874 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:51.222925 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:51.222927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.223602 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.223603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:51.224137 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:51.224139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.224499 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.224500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:51.224566 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:51.225045 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:51.225050 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:51.227198 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
W20260807 06:38:51.227347 24559 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:51.222429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:51833 (local address 127.11.114.254:43979)
0807 06:38:51.222495 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:51.222497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:51.222511 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:38:51.222605 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:51.222606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:51.222621 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:51.222667 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:51.223003 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.223518 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.224221 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.224425 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.224500 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:51.227284 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:51833: BlockingRecv error: recv got EOF from 127.11.114.195:51833 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 06:38:51 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:51 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:51.964819 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:51.960050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:46347)
0807 06:38:51.960216 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:51.960230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:51.960299 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:51.960440 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:51.960440 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:51.960604 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:51.960657 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:51.960658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.961409 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.961410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:51.961970 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:51.961971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.962328 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.962329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:51.962376 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:51.962841 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:51.962846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:51.964753 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
W20260807 06:38:51.965022 24563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:51.960137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:46347 (local address 127.11.114.254:43979)
0807 06:38:51.960293 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:51.960295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:51.960305 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:51.960348 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:51.960349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:51.960360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:51.960422 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:51.960744 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.961329 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.962033 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:51.962258 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:51.962316 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:51.964929 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:46347: BlockingRecv error: recv got EOF from 127.11.114.193:46347 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
Aug 07 06:38:52 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:52 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:52.103590 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:52.093920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:44867)
0807 06:38:52.094113 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:52.094125 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:52.094243 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:52.094404 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:52.094405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:52.094593 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:52.094644 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:52.094647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.095330 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.095331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:52.095870 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:52.095872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.096301 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.096302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:52.096383 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:52.101599 (+ 5216us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:52.101604 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:52.103530 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":100,"threads_started":1}
W20260807 06:38:52.103749 24563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:52.094025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:44867 (local address 127.11.114.254:43979)
0807 06:38:52.094096 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:52.094099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:52.094192 (+ 93us) server_negotiation.cc:407] Connection header received
0807 06:38:52.094313 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:52.094314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:52.094332 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:52.094397 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:52.094746 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.095242 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.095955 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.096207 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.096283 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:52.103646 (+ 7363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:44867: BlockingRecv error: recv got EOF from 127.11.114.194:44867 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 06:38:52 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:52 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:52.232602 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:52.227783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:58157)
0807 06:38:52.227956 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:52.227967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:52.228054 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:52.228224 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:52.228225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:52.228415 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:52.228466 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:52.228468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.229178 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.229180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:52.229796 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:52.229799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.230167 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.230168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:52.230235 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:52.230741 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:52.230746 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:52.232502 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
W20260807 06:38:52.232673 24563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:52.227868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:58157 (local address 127.11.114.254:43979)
0807 06:38:52.227937 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:52.227939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:52.228033 (+ 94us) server_negotiation.cc:407] Connection header received
0807 06:38:52.228126 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:52.228127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:52.228144 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:52.228217 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:52.228544 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.229066 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.229892 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.230093 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.230169 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:52.232603 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:58157: BlockingRecv error: recv got EOF from 127.11.114.195:58157 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 06:38:52 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:52 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:52.969858 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:52.965290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:44759)
0807 06:38:52.965455 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:52.965465 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:52.965562 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:52.965700 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:52.965701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:52.965866 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:52.965918 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:52.965920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.966550 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.966551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:52.967101 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:52.967103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.967417 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.967418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:52.967458 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:52.967894 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:52.967899 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:52.969802 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
W20260807 06:38:52.969992 24567 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:52.965329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:44759 (local address 127.11.114.254:43979)
0807 06:38:52.965451 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:52.965453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:52.965551 (+ 98us) server_negotiation.cc:407] Connection header received
0807 06:38:52.965627 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:52.965628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:52.965639 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:52.965697 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:52.965978 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.966479 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.967170 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:52.967358 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:52.967408 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:52.969887 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:44759: BlockingRecv error: recv got EOF from 127.11.114.193:44759 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
Aug 07 06:38:53 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:53 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:53.108888 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:53.104059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:48421)
0807 06:38:53.104230 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:53.104245 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:53.104319 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:53.104464 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:53.104465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:53.104651 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:53.104706 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:53.104708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.105368 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.105370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:53.106093 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:53.106097 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.106520 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.106521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:53.106581 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:53.107065 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:53.107071 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:53.108829 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
W20260807 06:38:53.108995 24567 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:53.104187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:48421 (local address 127.11.114.254:43979)
0807 06:38:53.104264 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:53.104266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:53.104307 (+ 41us) server_negotiation.cc:407] Connection header received
0807 06:38:53.104386 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:53.104387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:53.104398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:53.104441 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:53.104773 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.105266 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.106204 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.106432 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.106507 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:53.108923 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:48421: BlockingRecv error: recv got EOF from 127.11.114.194:48421 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 06:38:53 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:53 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:53.238206 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:53.233189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:46635)
0807 06:38:53.233328 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:53.233338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:53.233383 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:53.233500 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:53.233501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:53.233760 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:53.233816 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:53.233818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.234684 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.234685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:53.235300 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:53.235303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.235706 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.235707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:53.235761 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:53.236247 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:53.236252 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:53.238146 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260807 06:38:53.238293 24567 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:53.233279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:46635 (local address 127.11.114.254:43979)
0807 06:38:53.233358 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:53.233360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:53.233394 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:38:53.233425 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:53.233425 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:53.233438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:53.233482 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:53.234041 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.234620 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.235387 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.235637 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.235698 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:53.238231 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:46635: BlockingRecv error: recv got EOF from 127.11.114.195:46635 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 06:38:53 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:53 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:53.975422 24571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:53.970378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:46063)
0807 06:38:53.970539 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:53.970551 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:53.970611 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:53.970872 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:53.970874 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:53.971059 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:53.971114 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:53.971116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.971854 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.971855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:53.972487 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:53.972489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.972861 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.972862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:53.972915 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:53.973409 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:53.973415 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:53.975322 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
W20260807 06:38:53.975471 24575 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:53.970420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:46063 (local address 127.11.114.254:43979)
0807 06:38:53.970681 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:53.970683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:53.970698 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:38:53.970720 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:53.970721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:53.970732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:53.970806 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:53.971253 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.971768 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.972567 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:53.972785 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:53.972845 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:53.975399 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:46063: BlockingRecv error: recv got EOF from 127.11.114.193:46063 (error 108)
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":183,"threads_started":1}
Aug 07 06:38:54 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:54 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:54.114310 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:54.109418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:35977)
0807 06:38:54.109589 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:54.109602 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:54.109662 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:54.109793 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:54.109794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:54.109963 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:54.110014 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:54.110016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.110660 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.110661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:54.111246 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:54.111249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.111579 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.111579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:54.111622 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:54.112179 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:54.112184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:54.114234 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
W20260807 06:38:54.114398 24575 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:54.109463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:35977 (local address 127.11.114.254:43979)
0807 06:38:54.109647 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:54.109649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:54.109664 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:38:54.109716 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:54.109717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:54.109728 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:54.109780 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:54.110071 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.110585 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.111321 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.111514 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.111567 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:54.114333 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:35977: BlockingRecv error: recv got EOF from 127.11.114.194:35977 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
Aug 07 06:38:54 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:54 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:54.243913 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:54.238726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:54457)
0807 06:38:54.238912 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:54.238926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:54.239004 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:54.239155 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:54.239156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:54.239371 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:54.239421 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:54.239424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.240105 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.240106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:54.240679 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:54.240681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.241042 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.241043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:54.241106 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:54.241829 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:54.241834 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:54.243840 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
W20260807 06:38:54.244063 24575 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:54.238834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:54457 (local address 127.11.114.254:43979)
0807 06:38:54.238910 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:54.238912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:54.238982 (+ 70us) server_negotiation.cc:407] Connection header received
0807 06:38:54.239063 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:54.239064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:54.239078 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:54.239148 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:54.239505 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.240014 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.240768 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.240970 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.241042 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:54.243973 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:54457: BlockingRecv error: recv got EOF from 127.11.114.195:54457 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 06:38:54 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:54 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:54.980604 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:54.975930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:58897)
0807 06:38:54.976074 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:54.976084 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:54.976134 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:54.976309 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:54.976310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:54.976485 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:54.976542 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:54.976543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.977272 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.977273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:54.977881 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:54.977882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.978220 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.978221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:54.978266 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:54.978716 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:54.978721 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:54.980514 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
W20260807 06:38:54.980654 24579 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:54.975994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:58897 (local address 127.11.114.254:43979)
0807 06:38:54.976124 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:54.976126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:54.976135 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:54.976185 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:54.976186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:54.976196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:54.976265 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:54.976621 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.977192 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.977956 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:54.978156 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:54.978209 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:54.980600 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:58897: BlockingRecv error: recv got EOF from 127.11.114.193:58897 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
Aug 07 06:38:55 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:55 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:55.120163 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:55.114837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:41093)
0807 06:38:55.115032 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:55.115047 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:55.115138 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:55.115337 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:55.115338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:55.115585 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:55.115654 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:55.115657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.116477 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.116479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:55.117104 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:55.117106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.117578 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.117580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:55.117657 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:55.118148 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:55.118153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:55.120106 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":95,"threads_started":1}
W20260807 06:38:55.120311 24579 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:55.114922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:41093 (local address 127.11.114.254:43979)
0807 06:38:55.115014 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:55.115017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:55.115113 (+ 96us) server_negotiation.cc:407] Connection header received
0807 06:38:55.115215 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:55.115216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:55.115239 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:55.115311 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:55.115757 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.116373 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.117216 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.117439 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.117565 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:55.120224 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:41093: BlockingRecv error: recv got EOF from 127.11.114.194:41093 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 06:38:55 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:55 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:55.249336 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:55.244417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:50247)
0807 06:38:55.244663 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:55.244675 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:55.244769 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:55.244912 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:55.244913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:55.245115 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:55.245167 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:55.245169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.245928 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.245930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:55.246483 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:55.246484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.246839 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.246840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:55.246905 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:55.247371 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:55.247375 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:55.249237 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":138,"threads_started":1}
W20260807 06:38:55.249388 24579 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:55.244534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:50247 (local address 127.11.114.254:43979)
0807 06:38:55.244638 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:55.244641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:55.244745 (+ 104us) server_negotiation.cc:407] Connection header received
0807 06:38:55.244834 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:55.244834 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:55.244850 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:55.244895 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:55.245248 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.245835 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.246573 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.246768 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.246842 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:55.249321 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:50247: BlockingRecv error: recv got EOF from 127.11.114.195:50247 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 06:38:55 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:55 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:55.986472 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:55.981187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:35341)
0807 06:38:55.981339 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:55.981350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:55.981400 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:55.981679 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:55.981680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:55.981875 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:55.981927 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:55.981929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.982666 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.982666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:55.983332 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:55.983336 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.983816 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.983817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:55.983864 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:55.984388 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:55.984392 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:55.986366 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
W20260807 06:38:55.986527 24584 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:55.981369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:35341 (local address 127.11.114.254:43979)
0807 06:38:55.981506 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:55.981508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:55.981518 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:55.981573 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:55.981574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:55.981584 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:55.981663 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:55.982031 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.982588 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.983436 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:55.983749 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:55.983804 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:55.986452 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:35341: BlockingRecv error: recv got EOF from 127.11.114.193:35341 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
Aug 07 06:38:56 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:56 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:56.125766 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:56.120759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:36933)
0807 06:38:56.120934 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:56.120945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:56.121011 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:56.121161 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:56.121163 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:56.121350 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:56.121404 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:56.121405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.122275 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.122276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:56.122815 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:56.122817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.123140 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.123141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:56.123184 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:56.123628 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:56.123633 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:56.125705 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
W20260807 06:38:56.125906 24584 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:56.120859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:36933 (local address 127.11.114.254:43979)
0807 06:38:56.120934 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:56.120936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:56.121007 (+ 71us) server_negotiation.cc:407] Connection header received
0807 06:38:56.121082 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:56.121082 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:56.121094 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:56.121140 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:56.121482 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.122159 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.122890 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.123074 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.123125 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:56.125822 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:36933: BlockingRecv error: recv got EOF from 127.11.114.194:36933 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 06:38:56 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:56 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:56.254673 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:56.249947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:35869)
0807 06:38:56.250117 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:56.250128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:56.250196 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:56.250335 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:56.250335 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:56.250511 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:56.250564 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:56.250566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.251221 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.251221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:56.251765 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:56.251767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.252107 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.252108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:56.252148 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:56.252685 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:56.252699 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:56.254603 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
W20260807 06:38:56.254810 24584 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:56.250048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:35869 (local address 127.11.114.254:43979)
0807 06:38:56.250123 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:56.250125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:56.250185 (+ 60us) server_negotiation.cc:407] Connection header received
0807 06:38:56.250241 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:56.250241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:56.250255 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:56.250312 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:56.250630 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.251128 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.251849 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.252039 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.252094 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:56.254725 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:35869: BlockingRecv error: recv got EOF from 127.11.114.195:35869 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 06:38:56 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:56 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:56.992198 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:56.987006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:48967)
0807 06:38:56.987182 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:56.987199 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:56.987274 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:56.987457 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:56.987458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:56.987639 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:56.987695 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:56.987696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.988395 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.988396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:56.989218 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:56.989237 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.989763 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.989764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:56.989814 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:56.990271 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:56.990276 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:56.992130 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":66,"threads_started":1}
W20260807 06:38:56.992393 24588 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:56.987175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:48967 (local address 127.11.114.254:43979)
0807 06:38:56.987323 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:56.987324 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:56.987333 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:38:56.987356 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:56.987356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:56.987367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:56.987436 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:56.987767 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.988299 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.989332 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:56.989681 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:56.989741 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:56.992295 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:48967: BlockingRecv error: recv got EOF from 127.11.114.193:48967 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
Aug 07 06:38:57 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:57 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:57.134927 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:57.126242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:41369)
0807 06:38:57.126436 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:57.126449 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:57.126538 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:57.126681 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:57.126682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:57.126868 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:57.126922 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:57.126923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.127589 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.127590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:57.128164 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:57.128166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.128500 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.128501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:57.128558 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:57.132705 (+ 4147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:57.132710 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:57.134858 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
W20260807 06:38:57.135056 24588 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:57.126389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:41369 (local address 127.11.114.254:43979)
0807 06:38:57.126444 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:57.126446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:57.126504 (+ 58us) server_negotiation.cc:407] Connection header received
0807 06:38:57.126591 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:57.126592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:57.126603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:57.126650 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:57.126994 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.127497 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.128244 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.128426 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.128476 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:57.134957 (+ 6481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:41369: BlockingRecv error: recv got EOF from 127.11.114.194:41369 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 06:38:57 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:57 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:57.260664 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:57.255214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:60555)
0807 06:38:57.255428 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:57.255441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:57.255546 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:57.255736 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:57.255737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:57.255939 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:57.255992 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:57.255994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.256767 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.256768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:57.257381 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:57.257383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.257862 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.257863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:57.257937 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:57.258485 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:57.258491 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:57.260553 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
W20260807 06:38:57.260725 24588 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:57.255299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:60555 (local address 127.11.114.254:43979)
0807 06:38:57.255373 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:57.255377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:57.255522 (+ 145us) server_negotiation.cc:407] Connection header received
0807 06:38:57.255625 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:57.255626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:57.255647 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:57.255722 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:57.256082 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.256660 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.257475 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.257763 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.257861 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:57.260651 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:60555: BlockingRecv error: recv got EOF from 127.11.114.195:60555 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 06:38:57 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:57 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:57.998013 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:57.992948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:40673)
0807 06:38:57.993124 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:57.993136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:57.993226 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:57.993376 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:57.993376 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:57.993581 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:57.993634 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:57.993635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.994295 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.994296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:57.994838 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:57.994840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.995182 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.995183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:57.995228 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:57.995848 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:57.995854 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:57.997949 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
W20260807 06:38:57.998150 24593 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:57.993031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:40673 (local address 127.11.114.254:43979)
0807 06:38:57.993171 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:57.993173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:57.993213 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:38:57.993287 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:57.993287 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:57.993299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:57.993351 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:57.993693 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.994197 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.994918 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:57.995112 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:57.995163 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:57.998040 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:40673: BlockingRecv error: recv got EOF from 127.11.114.193:40673 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
Aug 07 06:38:58 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:58 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:58.140478 24594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:58.135466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:47737)
0807 06:38:58.135646 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:58.135662 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:58.135751 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:58.135901 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:58.135902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:58.136129 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:58.136184 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:58.136186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.136996 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.136997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:58.137612 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:58.137615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.137989 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.137990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:58.138050 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:58.138532 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:58.138537 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:58.140382 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
W20260807 06:38:58.140537 24593 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:58.135616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:47737 (local address 127.11.114.254:43979)
0807 06:38:58.135687 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:58.135689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:58.135725 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:38:58.135810 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:58.135811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:58.135822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:58.135873 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:58.136271 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.136913 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.137713 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.137913 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.137991 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:58.140472 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:47737: BlockingRecv error: recv got EOF from 127.11.114.194:47737 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 06:38:58 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:58 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:58.266561 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:58.261303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:49371)
0807 06:38:58.261475 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:58.261487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:58.261621 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:58.261849 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:58.261851 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:58.262037 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:58.262087 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:58.262089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.262779 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.262780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:58.263316 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:58.263318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.263703 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.263704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:58.263784 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:58.264389 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:58.264394 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:58.266497 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
W20260807 06:38:58.266644 24593 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:58.261477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:49371 (local address 127.11.114.254:43979)
0807 06:38:58.261578 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:58.261580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:58.261588 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:38:58.261727 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:58.261728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:58.261744 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:58.261825 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:58.262167 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.262687 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.263402 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.263638 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.263704 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:58.266593 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:49371: BlockingRecv error: recv got EOF from 127.11.114.195:49371 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 06:38:58 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:58 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:59.003711 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:58.998621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:59661)
0807 06:38:58.998790 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:58.998804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:58.998893 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:58.999039 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:58.999040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:58.999228 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:58.999281 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:58.999283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.999918 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:58.999919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:59.000703 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:59.000707 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.001085 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.001085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:59.001131 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:59.001697 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:59.001703 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:59.003638 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
W20260807 06:38:59.003852 24597 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:58.998733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:59661 (local address 127.11.114.254:43979)
0807 06:38:58.998878 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:58.998880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:58.998890 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:38:58.998941 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:58.998942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:58.998953 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:58.999012 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:58.999346 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:58.999846 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.000784 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.001004 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.001075 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:59.003737 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:59661: BlockingRecv error: recv got EOF from 127.11.114.193:59661 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
Aug 07 06:38:59 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:59 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:59.146560 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:59.141145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:33813)
0807 06:38:59.141324 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:59.141335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:59.141400 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:59.141610 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:59.141612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:59.141818 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:59.141872 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:59.141874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.142583 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.142584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:59.143169 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:59.143172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.143536 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.143536 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:59.143579 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:59.144277 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:59.144283 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:59.146492 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
W20260807 06:38:59.146718 24597 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:59.141239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:33813 (local address 127.11.114.254:43979)
0807 06:38:59.141329 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:59.141331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:59.141390 (+ 59us) server_negotiation.cc:407] Connection header received
0807 06:38:59.141453 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:59.141454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:59.141468 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:59.141521 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:59.141956 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.142502 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.143262 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.143458 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.143515 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:59.146632 (+ 3117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:33813: BlockingRecv error: recv got EOF from 127.11.114.194:33813 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 06:38:59 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:38:59 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:38:59.271757 24600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:38:59.267111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:39483)
0807 06:38:59.267275 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:38:59.267286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:38:59.267343 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:38:59.267488 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:38:59.267489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:38:59.267667 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:38:59.267717 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:59.267718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.268359 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.268360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:59.268909 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:38:59.268910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.269248 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.269249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:38:59.269327 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:59.269812 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:38:59.269816 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:38:59.271682 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":114,"threads_started":1}
W20260807 06:38:59.271868 24597 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:38:59.267177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:39483 (local address 127.11.114.254:43979)
0807 06:38:59.267246 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:38:59.267248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:38:59.267341 (+ 93us) server_negotiation.cc:407] Connection header received
0807 06:38:59.267385 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:38:59.267386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:38:59.267400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:38:59.267462 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:38:59.267782 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.268289 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.268980 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:38:59.269185 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:38:59.269239 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:38:59.271773 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:39483: BlockingRecv error: recv got EOF from 127.11.114.195:39483 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 06:39:00 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:00 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:00.009320 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:00.004260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:58973)
0807 06:39:00.004458 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:00.004469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:00.004535 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:00.004689 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:00.004690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:00.004866 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:00.004915 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:00.004916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.005706 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.005707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:00.006274 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:00.006276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.006654 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.006655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:00.006696 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:00.007226 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:00.007241 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:00.009247 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":123,"threads_started":1}
W20260807 06:39:00.009490 24604 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:00.004342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:58973 (local address 127.11.114.254:43979)
0807 06:39:00.004468 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:00.004470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:00.004543 (+ 73us) server_negotiation.cc:407] Connection header received
0807 06:39:00.004575 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:00.004591 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:00.004604 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:00.004673 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:00.005000 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.005628 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.006369 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.006584 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.006645 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:00.009393 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:58973: BlockingRecv error: recv got EOF from 127.11.114.193:58973 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
Aug 07 06:39:00 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:00 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:00.151721 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:00.147028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:38693)
0807 06:39:00.147191 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:00.147202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:00.147260 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:00.147404 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:00.147405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:00.147595 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:00.147647 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:00.147648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.148284 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.148285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:00.148845 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:00.148847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.149182 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.149183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:00.149226 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:00.149820 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:00.149825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:00.151651 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
W20260807 06:39:00.151939 24604 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:00.147101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:38693 (local address 127.11.114.254:43979)
0807 06:39:00.147149 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:00.147151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:00.147257 (+ 106us) server_negotiation.cc:407] Connection header received
0807 06:39:00.147320 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:00.147321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:00.147333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:00.147383 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:00.147707 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.148212 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.148924 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.149114 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.149168 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:00.151860 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:38693: BlockingRecv error: recv got EOF from 127.11.114.194:38693 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 07 06:39:00 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:00 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:00.277649 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:00.272298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:45415)
0807 06:39:00.272488 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:00.272509 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:00.272593 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:00.272787 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:00.272789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:00.272996 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:00.273093 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:00.273095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.273823 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.273824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:00.274447 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:00.274449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.274859 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.274860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:00.274937 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:00.275581 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:00.275592 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:00.277571 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":131,"threads_started":1}
W20260807 06:39:00.277745 24604 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:00.272380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:45415 (local address 127.11.114.254:43979)
0807 06:39:00.272448 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:00.272450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:00.272579 (+ 129us) server_negotiation.cc:407] Connection header received
0807 06:39:00.272653 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:00.272654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:00.272683 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:00.272755 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:00.273168 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.273734 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.274526 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:00.274738 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:00.274807 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:00.277662 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:45415: BlockingRecv error: recv got EOF from 127.11.114.195:45415 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 06:39:01 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:01 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:01.015012 24610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:01.009946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:43179)
0807 06:39:01.010136 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:01.010150 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:01.010256 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:01.010418 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:01.010419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:01.010638 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:01.010691 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:01.010693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.011461 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.011462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:01.012057 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:01.012059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.012495 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.012496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:01.012563 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:01.013035 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:01.013040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:01.014915 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":91,"threads_started":1}
W20260807 06:39:01.015084 24611 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:01.009966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:43179 (local address 127.11.114.254:43979)
0807 06:39:01.010102 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:01.010105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:01.010232 (+ 127us) server_negotiation.cc:407] Connection header received
0807 06:39:01.010316 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:01.010317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:01.010334 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:01.010399 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:01.010781 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.011362 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.012178 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.012410 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.012497 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:01.015009 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:43179: BlockingRecv error: recv got EOF from 127.11.114.193:43179 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
Aug 07 06:39:01 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:01 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:01.157892 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:01.152397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:49927)
0807 06:39:01.152563 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:01.152579 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:01.152686 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:01.152878 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:01.152879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:01.153101 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:01.153153 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:01.153154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.153985 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.153986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:01.154586 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:01.154588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.155010 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.155012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:01.155091 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:01.155621 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:01.155626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:01.157831 (+ 2205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
W20260807 06:39:01.158058 24611 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:01.152514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:49927 (local address 127.11.114.254:43979)
0807 06:39:01.152580 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:01.152583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:01.152679 (+ 96us) server_negotiation.cc:407] Connection header received
0807 06:39:01.152771 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:01.152772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:01.152787 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:01.152871 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:01.153250 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.153890 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.154690 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.154912 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.155008 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:01.157961 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:49927: BlockingRecv error: recv got EOF from 127.11.114.194:49927 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 06:39:01 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:01 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:01.283720 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:01.278228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:51661)
0807 06:39:01.278361 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:01.278374 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:01.278451 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:01.279020 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:01.279021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:01.279249 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:01.279309 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:01.279311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.279970 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.279971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:01.280616 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:01.280618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.280979 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.280979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:01.281022 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:01.281500 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:01.281506 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:01.283615 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
W20260807 06:39:01.283918 24611 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:01.278302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:51661 (local address 127.11.114.254:43979)
0807 06:39:01.278365 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:01.278367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:01.278429 (+ 62us) server_negotiation.cc:407] Connection header received
0807 06:39:01.278546 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:01.278547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:01.278565 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:01.278637 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:01.279395 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.279892 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.280712 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:01.280907 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:01.280970 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:01.283795 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:51661: BlockingRecv error: recv got EOF from 127.11.114.195:51661 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 06:39:02 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:02 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:02.020787 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:02.015701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:37049)
0807 06:39:02.015862 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:02.015885 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:02.015944 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:02.016118 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:02.016119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:02.016348 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:02.016409 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:02.016411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.017157 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.017158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:02.017767 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:02.017769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.018203 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.018204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:02.018271 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:02.018785 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:02.018790 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:02.020726 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
W20260807 06:39:02.020963 24618 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:02.015789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:37049 (local address 127.11.114.254:43979)
0807 06:39:02.015965 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:02.015970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:02.015984 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:39:02.016010 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:02.016010 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:02.016022 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:02.016098 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:02.016493 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.017057 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.017868 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.018102 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.018173 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:02.020867 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:37049: BlockingRecv error: recv got EOF from 127.11.114.193:37049 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
Aug 07 06:39:02 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:02 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:02.169016 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:02.158453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:39525)
0807 06:39:02.158764 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:02.158778 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:02.158853 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:02.158992 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:02.158994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:02.159195 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:02.159248 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:02.159250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.159932 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.159933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:02.160504 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:02.160506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.160888 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.160889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:02.160952 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:02.166828 (+ 5876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:02.166834 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:02.168941 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
W20260807 06:39:02.169137 24618 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:02.158581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:39525 (local address 127.11.114.254:43979)
0807 06:39:02.158771 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:02.158774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:02.158830 (+ 56us) server_negotiation.cc:407] Connection header received
0807 06:39:02.158912 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:02.158913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:02.158926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:02.158981 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:02.159338 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.159849 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.160596 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.160803 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.160877 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:02.169042 (+ 8165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:39525: BlockingRecv error: recv got EOF from 127.11.114.194:39525 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Aug 07 06:39:02 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:02 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.195@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:02.289470 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:02.284326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.195:36091)
0807 06:39:02.284518 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:02.284533 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:02.284630 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:02.284803 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:02.284804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:02.285005 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:02.285052 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:02.285053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.285892 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.285894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:02.286473 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:02.286475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.286841 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.286841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:02.286901 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:02.287381 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:02.287385 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:02.289370 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
W20260807 06:39:02.289523 24618 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:02.284452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:36091 (local address 127.11.114.254:43979)
0807 06:39:02.284522 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:02.284525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:02.284609 (+ 84us) server_negotiation.cc:407] Connection header received
0807 06:39:02.284701 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:02.284702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:02.284722 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:02.284794 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:02.285136 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.285803 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.286562 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:02.286767 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:02.286843 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:02.289456 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.195:36091: BlockingRecv error: recv got EOF from 127.11.114.195:36091 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 06:39:03 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:03 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.193@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:03.026607 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.021440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.193:45835)
0807 06:39:03.021658 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:03.021670 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:03.021734 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:03.021896 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:03.021896 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:03.022080 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:03.022170 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.022172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.023095 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.023096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.023679 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.023681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.024033 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.024034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.024076 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.024531 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:03.024536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:03.026516 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":129,"threads_started":1}
W20260807 06:39:03.026661 24627 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:03.021586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:45835 (local address 127.11.114.254:43979)
0807 06:39:03.021729 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:03.021731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:03.021743 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:39:03.021799 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:03.021800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:03.021812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:03.021874 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:03.022289 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.023000 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.023764 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.023965 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.024023 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.026605 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.193:45835: BlockingRecv error: recv got EOF from 127.11.114.193:45835 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
Aug 07 06:39:03 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:39:03 dist-test-slave-gkw7 krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.114.194@KRBTEST.COM for oryx/127.11.114.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:39:03.174680 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.169518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43979 (local address 127.11.114.194:56891)
0807 06:39:03.169713 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:03.169726 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:03.169820 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:03.169988 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:03.169989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:03.170241 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:03.170296 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.170298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.171139 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.171140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.171752 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.171754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.172195 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.172196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.172262 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.172708 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:03.172713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:03.174620 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.114.254:43979: Server oryx/127.11.114.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
W20260807 06:39:03.174824 24627 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:03.169600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:56891 (local address 127.11.114.254:43979)
0807 06:39:03.169695 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:03.169698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:03.169799 (+ 101us) server_negotiation.cc:407] Connection header received
0807 06:39:03.169897 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:03.169898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:03.169912 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:03.169977 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:03.170386 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.171018 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.171846 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.172096 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.172193 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.174729 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.114.194:56891: BlockingRecv error: recv got EOF from 127.11.114.194:56891 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 06:39:03.208818 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 23933
I20260807 06:39:03.218655 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 24150
I20260807 06:39:03.224951 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 24355
I20260807 06:39:03.232012 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 23826
2026-08-07T06:39:03Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17734 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T06:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:39:03Z Disabled control of system clock
I20260807 06:39:03.266244 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:43017
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:33279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:43017
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 06:39:03.353440 24637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:03.353729 24637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:03.353750 24637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:03.354931 24637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:39:03.354972 24637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:03.354985 24637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:03.355034 24637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:39:03.355060 24637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:39:03.356462 24637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:43017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:43017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:03.356721 24637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:03.357025 24637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:39:03.362313 24637 server_base.cc:1034] running on GCE node
W20260807 06:39:03.362306 24645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:03.362286 24643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:03.362298 24642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:03.362818 24637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:03.363116 24637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:03.364256 24637 hybrid_clock.cc:648] HybridClock initialized: now 1786084743364245 us; error 26 us; skew 500 ppm
I20260807 06:39:03.365592 24637 webserver.cc:460] Webserver started at http://127.11.114.254:42167/ using document root <none> and password file <none>
I20260807 06:39:03.365829 24637 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:03.365870 24637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:03.365979 24637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:03.367022 24637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "7b7ba55accda4a17ae33023e940abb44"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "2fa8773d03d426fb8c48e550a88c141a"
server_key_iv: "f495d4264e143c98f0c3a85cb4182b54"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.367327 24637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "7b7ba55accda4a17ae33023e940abb44"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "2fa8773d03d426fb8c48e550a88c141a"
server_key_iv: "f495d4264e143c98f0c3a85cb4182b54"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.368526 24637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:39:03.369185 24654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.369359 24637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:39:03.369436 24637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "7b7ba55accda4a17ae33023e940abb44"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "2fa8773d03d426fb8c48e550a88c141a"
server_key_iv: "f495d4264e143c98f0c3a85cb4182b54"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.369490 24637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:03.389731 24637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:03.389994 24637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:03.390107 24637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:03.394014 24637 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:43017
I20260807 06:39:03.394068 24736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:43017 every 8 connection(s)
I20260807 06:39:03.394444 24637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:39:03.395174 24737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:03.397914 24737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44: Bootstrap starting.
I20260807 06:39:03.398538 24737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:03.398847 24737 log.cc:824] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:03.399549 24737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44: No bootstrap required, opened a new log
I20260807 06:39:03.401206 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 24637
I20260807 06:39:03.401120 24737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ba55accda4a17ae33023e940abb44" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43017 } }
I20260807 06:39:03.401281 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:39:03.401279 24737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:03.401331 24737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7ba55accda4a17ae33023e940abb44, State: Initialized, Role: FOLLOWER
I20260807 06:39:03.401482 11723 external_mini_cluster.cc:1428] Setting key 05825d1729fe0cd1a662cf7a82a63e30
I20260807 06:39:03.401518 24737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ba55accda4a17ae33023e940abb44" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43017 } }
I20260807 06:39:03.401634 24737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:39:03.401693 24737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:39:03.401749 24737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:03.402496 24737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ba55accda4a17ae33023e940abb44" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43017 } }
I20260807 06:39:03.402585 24737 leader_election.cc:302] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b7ba55accda4a17ae33023e940abb44; no voters:
I20260807 06:39:03.402805 24737 leader_election.cc:288] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:39:03.402834 24743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:39:03.403000 24743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [term 1 LEADER]: Becoming Leader. State: Replica: 7b7ba55accda4a17ae33023e940abb44, State: Running, Role: LEADER
I20260807 06:39:03.403133 24743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ba55accda4a17ae33023e940abb44" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43017 } }
I20260807 06:39:03.403126 24737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:39:03.403529 24744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b7ba55accda4a17ae33023e940abb44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ba55accda4a17ae33023e940abb44" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43017 } } }
I20260807 06:39:03.403563 24745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b7ba55accda4a17ae33023e940abb44. Latest consensus state: current_term: 1 leader_uuid: "7b7ba55accda4a17ae33023e940abb44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ba55accda4a17ae33023e940abb44" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43017 } } }
I20260807 06:39:03.403741 24745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [sys.catalog]: This master's current role is: LEADER
I20260807 06:39:03.403944 24748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:39:03.403662 24744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44 [sys.catalog]: This master's current role is: LEADER
I20260807 06:39:03.404492 24748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:39:03.406335 24748 catalog_manager.cc:1347] Generated new cluster ID: d1c9884c7fa247f4919abd3da2831dbf
I20260807 06:39:03.406392 24748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:39:03.408044 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.402158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.11.114.254:43017)
0807 06:39:03.402365 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:03.402380 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:03.402405 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:39:03.402498 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:03.402504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:03.402607 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:03.402750 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:03.403005 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.404767 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.405886 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.406497 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.406620 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.407357 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:03.407362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:03.407362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:39:03.407564 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:03.407605 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:39:03.407620 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:03.407694 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:03.407761 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:39:03.407842 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260807 06:39:03.421870 24748 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:39:03.422595 24748 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:39:03.424986 24748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b7ba55accda4a17ae33023e940abb44: Generated new TSK 0
I20260807 06:39:03.425153 24748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:39:03.426842 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43017
--builtin_ntp_servers=127.11.114.212:33279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 06:39:03.511307 24769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:03.511554 24769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:03.511602 24769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:03.512820 24769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:03.512887 24769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:03.512930 24769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:39:03.514374 24769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:03.514644 24769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:03.514956 24769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:03.521019 24778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:03.521010 24779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:03.520991 24782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:03.521318 24769 server_base.cc:1034] running on GCE node
I20260807 06:39:03.521600 24769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:03.521909 24769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:03.523092 24769 hybrid_clock.cc:648] HybridClock initialized: now 1786084743523067 us; error 40 us; skew 500 ppm
I20260807 06:39:03.524571 24769 webserver.cc:460] Webserver started at http://127.11.114.193:42269/ using document root <none> and password file <none>
I20260807 06:39:03.524847 24769 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:03.524919 24769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:03.525089 24769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:03.526250 24769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "e800015e803c4588ab0c2a76327549fc"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "af280a6fe527a8890c3d7d962a5197b1"
server_key_iv: "42aa30920149bb3453a01dad609c493f"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.526666 24769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "e800015e803c4588ab0c2a76327549fc"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "af280a6fe527a8890c3d7d962a5197b1"
server_key_iv: "42aa30920149bb3453a01dad609c493f"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.528103 24769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:39:03.528895 24791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.529054 24769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:39:03.529165 24769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "e800015e803c4588ab0c2a76327549fc"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "af280a6fe527a8890c3d7d962a5197b1"
server_key_iv: "42aa30920149bb3453a01dad609c493f"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.529239 24769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:03.540107 24769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:03.540396 24769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:03.540542 24769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:03.540781 24769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:03.541124 24769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:03.541160 24769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.541185 24769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:03.541198 24769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.547557 24769 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:37497
I20260807 06:39:03.547627 24944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:37497 every 8 connection(s)
I20260807 06:39:03.547981 24769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:39:03.552560 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 24769
I20260807 06:39:03.552661 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:39:03.552865 11723 external_mini_cluster.cc:1428] Setting key 85022045cf0d82a3261757bc007bbd9b
I20260807 06:39:03.553297 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.548757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:44481 (local address 127.11.114.254:43017)
0807 06:39:03.548832 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:03.548834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:03.549080 (+ 246us) server_negotiation.cc:407] Connection header received
0807 06:39:03.549608 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:03.549611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:03.549629 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:03.549683 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:03.550346 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.550852 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.551791 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.551979 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.552030 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.552945 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:03.552946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:03.552946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:39:03.552951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:03.552967 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:39:03.552972 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:03.553033 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:03.553144 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 06:39:03.553197 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:39:03.553476 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.548761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43017 (local address 127.11.114.193:44481)
0807 06:39:03.548940 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:03.548973 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:03.549449 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:03.549739 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:03.549742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:03.549750 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:03.550269 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.550273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.550932 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.550933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.551718 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.551720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.552055 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.552056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.552188 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.552796 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:39:03.552796 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:03.552824 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:39:03.552846 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:03.553036 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:03.553041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:03.553043 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:39:03.553136 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 06:39:03.553265 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260807 06:39:03.553963 24946 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43017
I20260807 06:39:03.553997 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43017
--builtin_ntp_servers=127.11.114.212:33279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 06:39:03.554075 24946 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:03.554272 24946 heartbeater.cc:511] Master 127.11.114.254:43017 requested a full tablet report, sending...
I20260807 06:39:03.554816 24679 ts_manager.cc:194] Registered new tserver with Master: e800015e803c4588ab0c2a76327549fc (127.11.114.193:37497)
I20260807 06:39:03.555413 24679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.193:44481
W20260807 06:39:03.639137 24950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:03.639372 24950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:03.639397 24950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:03.640654 24950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:03.640699 24950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:03.640727 24950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:39:03.642256 24950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:03.642529 24950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:03.642863 24950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:03.648038 24963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:03.647998 24959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:03.648066 24960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:03.648200 24950 server_base.cc:1034] running on GCE node
I20260807 06:39:03.648571 24950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:03.648829 24950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:03.649991 24950 hybrid_clock.cc:648] HybridClock initialized: now 1786084743649967 us; error 40 us; skew 500 ppm
I20260807 06:39:03.651203 24950 webserver.cc:460] Webserver started at http://127.11.114.194:38791/ using document root <none> and password file <none>
I20260807 06:39:03.651453 24950 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:03.651525 24950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:03.651672 24950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:03.652767 24950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "89a9b85d2f704692b98b3a7d75df400b"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "43b3488e06344003a6955ea09f385e7c"
server_key_iv: "64034f1556abc77869807d85d30b129b"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.653129 24950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "89a9b85d2f704692b98b3a7d75df400b"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "43b3488e06344003a6955ea09f385e7c"
server_key_iv: "64034f1556abc77869807d85d30b129b"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.654563 24950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:39:03.655268 24971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.655441 24950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:39:03.655544 24950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "89a9b85d2f704692b98b3a7d75df400b"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "43b3488e06344003a6955ea09f385e7c"
server_key_iv: "64034f1556abc77869807d85d30b129b"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.655617 24950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:03.668334 24950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:03.668624 24950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:03.668771 24950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:03.668994 24950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:03.669376 24950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:03.669428 24950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.669471 24950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:03.669497 24950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.675701 24950 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:37541
I20260807 06:39:03.675762 25126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:37541 every 8 connection(s)
I20260807 06:39:03.676110 24950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:39:03.679584 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 24950
I20260807 06:39:03.679674 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:39:03.679876 11723 external_mini_cluster.cc:1428] Setting key 699962a42c1e6a298cbf748ab5127456
I20260807 06:39:03.680943 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43017
--builtin_ntp_servers=127.11.114.212:33279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 06:39:03.681825 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.676855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:44961 (local address 127.11.114.254:43017)
0807 06:39:03.676926 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:03.676929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:03.677182 (+ 253us) server_negotiation.cc:407] Connection header received
0807 06:39:03.677756 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:03.677758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:03.677769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:03.677824 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:03.678466 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.678956 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.680258 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.680571 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.680641 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.681568 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:03.681569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:03.681570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:39:03.681574 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:03.681590 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:39:03.681594 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:03.681642 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:03.681732 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 06:39:03.681757 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:39:03.682108 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.676849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43017 (local address 127.11.114.194:44961)
0807 06:39:03.677068 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:03.677100 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:03.677653 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:03.677879 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:03.677882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:03.677891 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:03.678398 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.678403 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.679035 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.679036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.680170 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.680173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.680666 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.680667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.680832 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.681366 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:39:03.681366 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:03.681408 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:39:03.681437 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:03.681645 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:03.681650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:03.681653 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:39:03.681723 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:39:03.681847 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
I20260807 06:39:03.682644 25127 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43017
I20260807 06:39:03.682765 25127 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:03.682973 25127 heartbeater.cc:511] Master 127.11.114.254:43017 requested a full tablet report, sending...
I20260807 06:39:03.683387 24679 ts_manager.cc:194] Registered new tserver with Master: 89a9b85d2f704692b98b3a7d75df400b (127.11.114.194:37541)
I20260807 06:39:03.683768 24679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.194:44961
W20260807 06:39:03.764746 25130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:03.764946 25130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:03.764967 25130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:03.766252 25130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:03.766297 25130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:03.766322 25130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:39:03.767627 25130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:03.767882 25130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:03.768185 25130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:03.773258 25136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:03.773382 25139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:03.773515 25130 server_base.cc:1034] running on GCE node
W20260807 06:39:03.773306 25135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:03.773803 25130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:03.774058 25130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:03.775235 25130 hybrid_clock.cc:648] HybridClock initialized: now 1786084743775207 us; error 39 us; skew 500 ppm
I20260807 06:39:03.776461 25130 webserver.cc:460] Webserver started at http://127.11.114.195:42001/ using document root <none> and password file <none>
I20260807 06:39:03.776731 25130 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:03.776803 25130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:03.776962 25130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:03.778167 25130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "ae5c34feaeae4c6ba0ecda2d2c5406c1"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "7eb19d2a8b4f8832d0631670387f8ed1"
server_key_iv: "9e80eb14cf34cbb48f97e0762c9584ce"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.778543 25130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "ae5c34feaeae4c6ba0ecda2d2c5406c1"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "7eb19d2a8b4f8832d0631670387f8ed1"
server_key_iv: "9e80eb14cf34cbb48f97e0762c9584ce"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.779899 25130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:39:03.780697 25148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.780874 25130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:39:03.780983 25130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "ae5c34feaeae4c6ba0ecda2d2c5406c1"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "7eb19d2a8b4f8832d0631670387f8ed1"
server_key_iv: "9e80eb14cf34cbb48f97e0762c9584ce"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.781060 25130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:03.788903 25130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:03.789186 25130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:03.789333 25130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:03.789615 25130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:03.789956 25130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:03.790012 25130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.790052 25130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:03.790078 25130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.795940 25130 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:39369
I20260807 06:39:03.795976 25315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:39369 every 8 connection(s)
I20260807 06:39:03.796337 25130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:39:03.796970 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 25130
I20260807 06:39:03.797071 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:39:03.797302 11723 external_mini_cluster.cc:1428] Setting key 549bb700a165a218fa493c5a1255a4fb
I20260807 06:39:03.802253 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.797117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:51659 (local address 127.11.114.254:43017)
0807 06:39:03.797163 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:03.797165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:03.797431 (+ 266us) server_negotiation.cc:407] Connection header received
0807 06:39:03.798145 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:03.798146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:03.798158 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:03.798214 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:03.798873 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.799385 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.800419 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.800634 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.800721 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.801923 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:03.801925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:03.801925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:39:03.801930 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:03.801946 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:39:03.801957 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:03.802021 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:03.802122 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 06:39:03.802156 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:39:03.802457 25318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.797125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43017 (local address 127.11.114.195:51659)
0807 06:39:03.797304 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:03.797335 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:03.798043 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:03.798286 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:03.798290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:03.798300 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:03.798779 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.798784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.799487 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.799489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.800318 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:03.800320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:03.800737 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:03.800738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:03.800874 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:03.801754 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:39:03.801755 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:03.801792 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:39:03.801816 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:03.802021 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:03.802028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:03.802032 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:39:03.802124 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:39:03.802229 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260807 06:39:03.802866 25316 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43017
I20260807 06:39:03.802946 25316 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:03.803149 25316 heartbeater.cc:511] Master 127.11.114.254:43017 requested a full tablet report, sending...
I20260807 06:39:03.803547 24679 ts_manager.cc:194] Registered new tserver with Master: ae5c34feaeae4c6ba0ecda2d2c5406c1 (127.11.114.195:39369)
I20260807 06:39:03.803975 24679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.195:51659
I20260807 06:39:03.809203 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:39:03.811193 24742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:03.810725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34090 (local address 127.11.114.254:43017)
0807 06:39:03.810801 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:03.810803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:03.810896 (+ 93us) server_negotiation.cc:407] Connection header received
0807 06:39:03.810946 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:03.810947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:03.810957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:03.811007 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:03.811119 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34090: BlockingRecv error: recv got EOF from 127.0.0.1:34090 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:39:03.811298 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 24769
I20260807 06:39:03.822715 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 24950
I20260807 06:39:03.832597 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 25130
I20260807 06:39:03.841015 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 24637
2026-08-07T06:39:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (609 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T06:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:39:03Z Disabled control of system clock
I20260807 06:39:03.874825 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39793
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:39:03.957783 25335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:03.957986 25335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:03.958006 25335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:03.959266 25335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:39:03.959308 25335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:03.959321 25335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:03.959334 25335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:39:03.959347 25335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:39:03.960748 25335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:03.961017 25335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:03.961333 25335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:03.966339 25342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:03.966356 25344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:03.966356 25335 server_base.cc:1034] running on GCE node
W20260807 06:39:03.966328 25341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:03.966859 25335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:03.967216 25335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:03.968390 25335 hybrid_clock.cc:648] HybridClock initialized: now 1786084743968385 us; error 36 us; skew 500 ppm
I20260807 06:39:03.969759 25335 webserver.cc:460] Webserver started at http://127.11.114.254:39971/ using document root <none> and password file <none>
I20260807 06:39:03.970029 25335 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:03.970139 25335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:03.970300 25335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:03.971400 25335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "ac77858b733c448e9620e16b7b4f63d5"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "13fb9c1784f94582d0f96c9c251bbe5e"
server_key_iv: "225056189d9a8e5879ab2ed665032357"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.971772 25335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "ac77858b733c448e9620e16b7b4f63d5"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "13fb9c1784f94582d0f96c9c251bbe5e"
server_key_iv: "225056189d9a8e5879ab2ed665032357"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.973150 25335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:39:03.973958 25356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.974149 25335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:03.974264 25335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "ac77858b733c448e9620e16b7b4f63d5"
format_stamp: "Formatted at 2026-08-07 06:39:03 on dist-test-slave-gkw7"
server_key: "13fb9c1784f94582d0f96c9c251bbe5e"
server_key_iv: "225056189d9a8e5879ab2ed665032357"
server_key_version: "encryptionkey@0"
I20260807 06:39:03.974341 25335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:03.977928 25335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:03.978201 25335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:03.978353 25335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:03.982152 25335 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39793
I20260807 06:39:03.982187 25425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39793 every 8 connection(s)
I20260807 06:39:03.982524 25335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:39:03.983222 25427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:03.986037 25427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5: Bootstrap starting.
I20260807 06:39:03.986579 25427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:03.986817 25427 log.cc:824] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:03.987469 25427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5: No bootstrap required, opened a new log
I20260807 06:39:03.988993 25427 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac77858b733c448e9620e16b7b4f63d5" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39793 } }
I20260807 06:39:03.989130 25427 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:03.989188 25427 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac77858b733c448e9620e16b7b4f63d5, State: Initialized, Role: FOLLOWER
I20260807 06:39:03.989377 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 25335
I20260807 06:39:03.989355 25427 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac77858b733c448e9620e16b7b4f63d5" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39793 } }
I20260807 06:39:03.989439 25427 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:39:03.989476 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:39:03.989475 25427 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:39:03.989548 25427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:03.989724 11723 external_mini_cluster.cc:1428] Setting key 39d1b63daed36fa8fad346b60f319474
I20260807 06:39:03.990306 25427 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac77858b733c448e9620e16b7b4f63d5" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39793 } }
I20260807 06:39:03.990501 25427 leader_election.cc:302] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac77858b733c448e9620e16b7b4f63d5; no voters:
I20260807 06:39:03.990677 25427 leader_election.cc:288] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:39:03.990716 25433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:39:03.990912 25433 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [term 1 LEADER]: Becoming Leader. State: Replica: ac77858b733c448e9620e16b7b4f63d5, State: Running, Role: LEADER
I20260807 06:39:03.990947 25427 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:39:03.991052 25433 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac77858b733c448e9620e16b7b4f63d5" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39793 } }
I20260807 06:39:03.991717 25434 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac77858b733c448e9620e16b7b4f63d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac77858b733c448e9620e16b7b4f63d5" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39793 } } }
I20260807 06:39:03.991818 25434 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [sys.catalog]: This master's current role is: LEADER
I20260807 06:39:03.991827 25435 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac77858b733c448e9620e16b7b4f63d5. Latest consensus state: current_term: 1 leader_uuid: "ac77858b733c448e9620e16b7b4f63d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac77858b733c448e9620e16b7b4f63d5" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39793 } } }
I20260807 06:39:03.991907 25435 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5 [sys.catalog]: This master's current role is: LEADER
I20260807 06:39:03.992297 25441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:39:03.992702 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:03.990271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53920 (local address 127.11.114.254:39793)
0807 06:39:03.990477 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:03.990484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:03.990511 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:39:03.990641 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:03.990648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:03.990697 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:03.990870 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:03.992134 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:03.992138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:03.992139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:39:03.992144 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:03.992188 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:39:03.992209 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:03.992295 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:03.992427 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 06:39:03.992497 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260807 06:39:03.992884 25441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:39:03.994498 25441 catalog_manager.cc:1347] Generated new cluster ID: c98be146e21447ba802b36c53bb7c17c
I20260807 06:39:03.994577 25441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:39:04.006469 25441 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:39:04.006664 25441 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:39:04.014559 25441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac77858b733c448e9620e16b7b4f63d5: Generated new TSK 0
I20260807 06:39:04.014797 25441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:39:04.024382 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39793
--builtin_ntp_servers=127.11.114.212:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:39:04.108657 25457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:04.108935 25457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:04.108975 25457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:04.110342 25457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:04.110412 25457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:04.110458 25457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:39:04.111871 25457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:04.112170 25457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:04.112501 25457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:39:04.117856 25457 server_base.cc:1034] running on GCE node
W20260807 06:39:04.117794 25464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:04.117796 25462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:04.117794 25467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:04.118275 25457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:04.118533 25457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:04.119695 25457 hybrid_clock.cc:648] HybridClock initialized: now 1786084744119682 us; error 28 us; skew 500 ppm
I20260807 06:39:04.121012 25457 webserver.cc:460] Webserver started at http://127.11.114.193:43689/ using document root <none> and password file <none>
I20260807 06:39:04.121286 25457 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:04.121361 25457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:04.121518 25457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:04.122646 25457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "3fa9e930be194c3d9262e982bef8241a"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "c4fe7dd44d1a754989f396da7d1dbc6a"
server_key_iv: "f00a606afefca0fc6ff3d6f2142575ba"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.123035 25457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "3fa9e930be194c3d9262e982bef8241a"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "c4fe7dd44d1a754989f396da7d1dbc6a"
server_key_iv: "f00a606afefca0fc6ff3d6f2142575ba"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.124418 25457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:39:04.125247 25475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.125449 25457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:39:04.125581 25457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "3fa9e930be194c3d9262e982bef8241a"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "c4fe7dd44d1a754989f396da7d1dbc6a"
server_key_iv: "f00a606afefca0fc6ff3d6f2142575ba"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.125660 25457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:04.129093 25457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:04.129369 25457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:04.129515 25457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:04.129776 25457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:04.130041 25457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:04.130072 25457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.130092 25457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:04.130105 25457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.136210 25457 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:40203
I20260807 06:39:04.136344 25635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:40203 every 8 connection(s)
I20260807 06:39:04.136631 25457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:39:04.139585 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:04.137464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:44201 (local address 127.11.114.254:39793)
0807 06:39:04.137524 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:04.137526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:04.137797 (+ 271us) server_negotiation.cc:407] Connection header received
0807 06:39:04.138252 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:04.138253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:04.138262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:04.138316 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:04.139402 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:04.139403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:04.139403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:39:04.139408 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:04.139423 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:39:04.139427 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:04.139474 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:04.139518 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 06:39:04.139532 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:39:04.139830 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:04.137457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39793 (local address 127.11.114.193:44201)
0807 06:39:04.137671 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:04.137704 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:04.138162 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:04.138366 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:04.138369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:04.138377 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:04.139258 (+ 881us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:39:04.139259 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:04.139287 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:39:04.139316 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:04.139472 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:04.139474 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:04.139476 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:39:04.139526 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:39:04.139587 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260807 06:39:04.139878 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 25457
I20260807 06:39:04.139971 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:39:04.140192 11723 external_mini_cluster.cc:1428] Setting key eed457fe67305f63a3d9bcf057379640
I20260807 06:39:04.140422 25639 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39793
I20260807 06:39:04.140525 25639 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:04.140683 25639 heartbeater.cc:511] Master 127.11.114.254:39793 requested a full tablet report, sending...
I20260807 06:39:04.141243 25377 ts_manager.cc:194] Registered new tserver with Master: 3fa9e930be194c3d9262e982bef8241a (127.11.114.193:40203)
I20260807 06:39:04.141265 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39793
--builtin_ntp_servers=127.11.114.212:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:39:04.226723 25643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:04.227002 25643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:04.227053 25643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:04.228282 25643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:04.228360 25643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:04.228404 25643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:39:04.229915 25643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:04.230225 25643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:04.230578 25643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:04.236176 25649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:04.236182 25650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:04.236346 25652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:04.236356 25643 server_base.cc:1034] running on GCE node
I20260807 06:39:04.236749 25643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:04.237032 25643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:04.238229 25643 hybrid_clock.cc:648] HybridClock initialized: now 1786084744238155 us; error 97 us; skew 500 ppm
I20260807 06:39:04.239625 25643 webserver.cc:460] Webserver started at http://127.11.114.194:34109/ using document root <none> and password file <none>
I20260807 06:39:04.239912 25643 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:04.239987 25643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:04.240144 25643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:04.241250 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "49a30dd585d149a5b33a592c876c2d8a"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "246a9d47e0c79d912b4fccc1c630cbe3"
server_key_iv: "ac6ea907b01d2ac874edab9205d3d69c"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.241674 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "49a30dd585d149a5b33a592c876c2d8a"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "246a9d47e0c79d912b4fccc1c630cbe3"
server_key_iv: "ac6ea907b01d2ac874edab9205d3d69c"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.243110 25643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:39:04.244024 25660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.244248 25643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:39:04.244351 25643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "49a30dd585d149a5b33a592c876c2d8a"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "246a9d47e0c79d912b4fccc1c630cbe3"
server_key_iv: "ac6ea907b01d2ac874edab9205d3d69c"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.244433 25643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:04.248014 25643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:04.248307 25643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:04.248452 25643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:04.248677 25643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:04.249010 25643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:04.249073 25643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.249114 25643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:04.249140 25643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.255259 25643 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:36889
I20260807 06:39:04.255340 25806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:36889 every 8 connection(s)
I20260807 06:39:04.255678 25643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:39:04.257864 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 25643
I20260807 06:39:04.257984 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:39:04.258212 11723 external_mini_cluster.cc:1428] Setting key 0e40b76dcaedb7bb0165e6ebec1ae1c9
I20260807 06:39:04.259124 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:04.256484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:39803 (local address 127.11.114.254:39793)
0807 06:39:04.256553 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:04.256555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:04.256805 (+ 250us) server_negotiation.cc:407] Connection header received
0807 06:39:04.257272 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:04.257273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:04.257279 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:04.257345 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:04.258803 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:04.258804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:04.258804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:39:04.258809 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:04.258827 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:39:04.258832 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:04.258885 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:04.259015 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 06:39:04.259031 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:39:04.259351 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:04.256451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39793 (local address 127.11.114.194:39803)
0807 06:39:04.256660 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:04.256704 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:04.257171 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:04.257421 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:04.257424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:04.257433 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:04.258638 (+ 1205us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:39:04.258639 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:04.258678 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:39:04.258714 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:04.258906 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:04.258911 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:04.258914 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:39:04.259006 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:39:04.259075 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260807 06:39:04.259406 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39793
--builtin_ntp_servers=127.11.114.212:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 06:39:04.261242 25808 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39793
I20260807 06:39:04.261365 25808 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:04.261554 25808 heartbeater.cc:511] Master 127.11.114.254:39793 requested a full tablet report, sending...
I20260807 06:39:04.262015 25377 ts_manager.cc:194] Registered new tserver with Master: 49a30dd585d149a5b33a592c876c2d8a (127.11.114.194:36889)
W20260807 06:39:04.343709 25811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:04.343950 25811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:04.343989 25811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:04.345198 25811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:04.345263 25811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:04.345309 25811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:39:04.346738 25811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:04.347038 25811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:04.347371 25811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:04.352475 25822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:04.352475 25826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:04.352473 25828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:04.352770 25811 server_base.cc:1034] running on GCE node
I20260807 06:39:04.352977 25811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:04.353224 25811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:04.354367 25811 hybrid_clock.cc:648] HybridClock initialized: now 1786084744354342 us; error 41 us; skew 500 ppm
I20260807 06:39:04.355520 25811 webserver.cc:460] Webserver started at http://127.11.114.195:34003/ using document root <none> and password file <none>
I20260807 06:39:04.355770 25811 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:04.355818 25811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:04.355978 25811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:04.357036 25811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "e63aa83d21e244ed978edc259f2a92b4"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "56f5a4d1a0801ed9aa4548cc73ada710"
server_key_iv: "ee3015d1a2bf88177ae4b72b27f4cc74"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.357407 25811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "e63aa83d21e244ed978edc259f2a92b4"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "56f5a4d1a0801ed9aa4548cc73ada710"
server_key_iv: "ee3015d1a2bf88177ae4b72b27f4cc74"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.358744 25811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:39:04.359486 25836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.359694 25811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:39:04.359787 25811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "e63aa83d21e244ed978edc259f2a92b4"
format_stamp: "Formatted at 2026-08-07 06:39:04 on dist-test-slave-gkw7"
server_key: "56f5a4d1a0801ed9aa4548cc73ada710"
server_key_iv: "ee3015d1a2bf88177ae4b72b27f4cc74"
server_key_version: "encryptionkey@0"
I20260807 06:39:04.359861 25811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:04.363293 25811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:04.363570 25811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:04.363718 25811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:04.363926 25811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:04.364233 25811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:04.364265 25811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.364317 25811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:04.364346 25811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:04.370237 25811 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:35607
I20260807 06:39:04.370314 25988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:35607 every 8 connection(s)
I20260807 06:39:04.370639 25811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:39:04.373775 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:04.371379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:47727 (local address 127.11.114.254:39793)
0807 06:39:04.371443 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:04.371445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:04.371681 (+ 236us) server_negotiation.cc:407] Connection header received
0807 06:39:04.372124 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:04.372125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:04.372130 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:04.372187 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:04.373414 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:04.373415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:04.373415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:39:04.373418 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:04.373428 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:39:04.373432 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:04.373472 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:04.373654 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 06:39:04.373676 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:39:04.373816 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:04.371364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39793 (local address 127.11.114.195:47727)
0807 06:39:04.371555 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:04.371594 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:04.372042 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:04.372278 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:04.372282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:04.372289 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:04.373279 (+ 990us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:39:04.373280 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:04.373308 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:39:04.373337 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:04.373472 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:04.373474 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:04.373476 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:39:04.373584 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 06:39:04.373659 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260807 06:39:04.374310 25989 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39793
I20260807 06:39:04.374400 25989 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:04.374571 25989 heartbeater.cc:511] Master 127.11.114.254:39793 requested a full tablet report, sending...
I20260807 06:39:04.374943 25377 ts_manager.cc:194] Registered new tserver with Master: e63aa83d21e244ed978edc259f2a92b4 (127.11.114.195:35607)
I20260807 06:39:04.375051 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 25811
I20260807 06:39:04.375142 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:39:04.375386 11723 external_mini_cluster.cc:1428] Setting key 7cdf8efb8aaa34f3806f62e659878d3a
I20260807 06:39:04.376459 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:39:04.378607 25432 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:39:04.378033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53928 (local address 127.11.114.254:39793)
0807 06:39:04.378085 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:04.378087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:04.378260 (+ 173us) server_negotiation.cc:407] Connection header received
0807 06:39:04.378319 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:04.378320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:04.378325 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:04.378369 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:04.378520 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53928: BlockingRecv error: recv got EOF from 127.0.0.1:53928 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:39:04.378702 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 25457
I20260807 06:39:04.390136 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 25643
I20260807 06:39:04.413489 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 25811
I20260807 06:39:04.426440 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 25335
2026-08-07T06:39:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (584 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:39:04 dist-test-slave-gkw7 krb5kdc[26002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:39:04 dist-test-slave-gkw7 krb5kdc[26002](info): set up 2 sockets
Aug 07 06:39:04 dist-test-slave-gkw7 krb5kdc[26002](info): commencing operation
krb5kdc: starting...
W20260807 06:39:05.738401 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:39:05 dist-test-slave-gkw7 krb5kdc[26002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:39:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:39:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:39:05.865693 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:46565
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:46565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:39:05.948477 26019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:05.948702 26019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:05.948725 26019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:05.949829 26019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:39:05.949869 26019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:05.949884 26019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:05.949896 26019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:39:05.949909 26019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:39:05.951586 26019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:46565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:46565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:05.951787 26019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:05.951965 26019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:05.956964 26025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:05.956969 26027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:05.957197 26019 server_base.cc:1034] running on GCE node
W20260807 06:39:05.956986 26029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:05.957610 26019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:05.957978 26019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:05.959195 26019 hybrid_clock.cc:648] HybridClock initialized: now 1786084745959186 us; error 40 us; skew 500 ppm
Aug 07 06:39:05 dist-test-slave-gkw7 krb5kdc[26002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:05.962276 26019 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:39:05.962801 26019 webserver.cc:460] Webserver started at http://127.11.114.254:39109/ using document root <none> and password file <none>
I20260807 06:39:05.963107 26019 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:05.963195 26019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:05.963374 26019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:05.964632 26019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "5920d15178644da2abdb4366509eb351"
format_stamp: "Formatted at 2026-08-07 06:39:05 on dist-test-slave-gkw7"
server_key: "ff4a199160cd02c21d11dce5b1cb57c9"
server_key_iv: "7183188f261f88194c827960e3c8d620"
server_key_version: "encryptionkey@0"
I20260807 06:39:05.965090 26019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "5920d15178644da2abdb4366509eb351"
format_stamp: "Formatted at 2026-08-07 06:39:05 on dist-test-slave-gkw7"
server_key: "ff4a199160cd02c21d11dce5b1cb57c9"
server_key_iv: "7183188f261f88194c827960e3c8d620"
server_key_version: "encryptionkey@0"
I20260807 06:39:05.966831 26019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 06:39:05.967705 26036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:05.967904 26019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:39:05.968024 26019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "5920d15178644da2abdb4366509eb351"
format_stamp: "Formatted at 2026-08-07 06:39:05 on dist-test-slave-gkw7"
server_key: "ff4a199160cd02c21d11dce5b1cb57c9"
server_key_iv: "7183188f261f88194c827960e3c8d620"
server_key_version: "encryptionkey@0"
I20260807 06:39:05.968148 26019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:05.978852 26019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:05.981447 26019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:05.981675 26019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:05.985872 26019 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:46565
I20260807 06:39:05.985909 26103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:46565 every 8 connection(s)
I20260807 06:39:05.986301 26019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:39:05.987363 26104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:05.989693 26104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351: Bootstrap starting.
I20260807 06:39:05.990242 26104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:05.990479 26104 log.cc:824] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:05.991187 26104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351: No bootstrap required, opened a new log
I20260807 06:39:05.991377 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 26019
I20260807 06:39:05.991496 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:39:05.991694 11723 external_mini_cluster.cc:1428] Setting key d56033bb4ae728e8373bf6cf9be17de3
I20260807 06:39:05.993592 26104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5920d15178644da2abdb4366509eb351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46565 } }
I20260807 06:39:05.993784 26104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:05.993871 26104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5920d15178644da2abdb4366509eb351, State: Initialized, Role: FOLLOWER
I20260807 06:39:05.994022 26104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5920d15178644da2abdb4366509eb351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46565 } }
I20260807 06:39:05.994127 26104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:39:05.994172 26104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:39:05.994226 26104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:05.995391 26104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5920d15178644da2abdb4366509eb351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46565 } }
I20260807 06:39:05.995502 26104 leader_election.cc:302] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5920d15178644da2abdb4366509eb351; no voters:
I20260807 06:39:05.995672 26104 leader_election.cc:288] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:39:05.995793 26109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:39:05.995991 26104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:39:05.996047 26109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [term 1 LEADER]: Becoming Leader. State: Replica: 5920d15178644da2abdb4366509eb351, State: Running, Role: LEADER
I20260807 06:39:05.996239 26109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5920d15178644da2abdb4366509eb351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46565 } }
I20260807 06:39:05.996637 26111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5920d15178644da2abdb4366509eb351. Latest consensus state: current_term: 1 leader_uuid: "5920d15178644da2abdb4366509eb351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5920d15178644da2abdb4366509eb351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46565 } } }
I20260807 06:39:05.996634 26110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5920d15178644da2abdb4366509eb351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5920d15178644da2abdb4366509eb351" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46565 } } }
I20260807 06:39:05.996825 26111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [sys.catalog]: This master's current role is: LEADER
I20260807 06:39:05.996831 26110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351 [sys.catalog]: This master's current role is: LEADER
I20260807 06:39:05.997090 26123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:39:05.997699 26123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:39:05.999696 26123 catalog_manager.cc:1347] Generated new cluster ID: e6cae2968ba44e898e9afa8c5b19eef3
I20260807 06:39:05.999799 26123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:39:05 dist-test-slave-gkw7 krb5kdc[26002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:39:06.004297 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:05.992274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47062 (local address 127.11.114.254:46565)
0807 06:39:05.992486 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:05.992496 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:05.992534 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:39:05.993548 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:05.993560 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:05.993629 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:05.993776 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:05.994580 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:05.996066 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:05.997289 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:05.997976 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:05.998115 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.000778 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:06.000782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:06.000782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:06.000795 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:06.002710 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.003091 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.003094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.003094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.003226 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.003450 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.003451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.003451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.003715 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:06.003908 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.003977 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.004123 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260807 06:39:06.019408 26123 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:39:06.020174 26123 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:39:06.028270 26123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351: Generated new TSK 0
I20260807 06:39:06.028501 26123 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:39:06.060986 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46565
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:39:06.148131 26138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:06.148350 26138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:06.148375 26138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:06.149483 26138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:06.149518 26138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:06.149572 26138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:39:06.151075 26138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:06.151275 26138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:06.151459 26138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:06.156711 26144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:06.156759 26143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:06.156788 26147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:06.156774 26138 server_base.cc:1034] running on GCE node
I20260807 06:39:06.157128 26138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:06.157385 26138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:06.158555 26138 hybrid_clock.cc:648] HybridClock initialized: now 1786084746158546 us; error 37 us; skew 500 ppm
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:06.161147 26138 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:39:06.161649 26138 webserver.cc:460] Webserver started at http://127.11.114.193:32993/ using document root <none> and password file <none>
I20260807 06:39:06.161896 26138 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:06.161965 26138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:06.162132 26138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:06.163223 26138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "817dccc2978c4f49ac28752d4233f3be"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "24904f2a691066650809da188f79793d"
server_key_iv: "2c780e8155f6eaeafc887c7799f4269d"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.163594 26138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "817dccc2978c4f49ac28752d4233f3be"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "24904f2a691066650809da188f79793d"
server_key_iv: "2c780e8155f6eaeafc887c7799f4269d"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.164965 26138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:39:06.165824 26158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.166004 26138 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:39:06.166107 26138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "817dccc2978c4f49ac28752d4233f3be"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "24904f2a691066650809da188f79793d"
server_key_iv: "2c780e8155f6eaeafc887c7799f4269d"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.166191 26138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:06.173414 26138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:06.175297 26138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:06.175470 26138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:06.175698 26138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:06.176023 26138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:06.176077 26138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.176111 26138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:06.176139 26138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.182204 26138 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:43665
I20260807 06:39:06.182268 26306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:43665 every 8 connection(s)
I20260807 06:39:06.182585 26138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:39:06.187096 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 26138
I20260807 06:39:06.187175 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:39:06.187383 11723 external_mini_cluster.cc:1428] Setting key 0eba6500433a4c4f2223f032a5535317
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:39:06.193094 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.183392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:57283 (local address 127.11.114.254:46565)
0807 06:39:06.183466 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.183469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.183734 (+ 265us) server_negotiation.cc:407] Connection header received
0807 06:39:06.184244 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.184246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.184262 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.184325 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:06.185323 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.185944 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.186937 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.187233 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.187321 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.190022 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:06.190024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:06.190024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:06.190036 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:06.191544 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.192002 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.192004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.192004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.192079 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.192352 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.192354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.192354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.192483 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:06.192561 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.192864 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.192988 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:39:06.193234 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.183370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46565 (local address 127.11.114.193:57283)
0807 06:39:06.183597 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.183643 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:06.184132 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:06.184409 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:06.184412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:06.185006 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:06.185200 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.185205 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.186028 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.186029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.186836 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.186838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.187330 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.187331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.187482 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.188168 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:06.188178 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:06.189912 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:06.191670 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:06.191678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:06.191680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:06.191918 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:06.192163 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:06.192164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:06.192164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:06.192283 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:06.192552 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:06.192559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:06.192718 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 06:39:06.192826 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 06:39:06.193000 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":116,"threads_started":1}
I20260807 06:39:06.193802 26308 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46565
I20260807 06:39:06.193920 26308 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:06.194165 26308 heartbeater.cc:511] Master 127.11.114.254:46565 requested a full tablet report, sending...
I20260807 06:39:06.194762 26057 ts_manager.cc:194] Registered new tserver with Master: 817dccc2978c4f49ac28752d4233f3be (127.11.114.193:43665)
I20260807 06:39:06.195417 26057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:57283
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:39:06.216922 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46565
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:39:06.301872 26315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:06.302098 26315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:06.302124 26315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:06.303193 26315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:06.303229 26315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:06.303256 26315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:39:06.304630 26315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:06.304816 26315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:06.304994 26315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:06.310166 26320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:06.310158 26323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:06.310309 26315 server_base.cc:1034] running on GCE node
W20260807 06:39:06.310158 26321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:06.310561 26315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:06.310770 26315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:06.311932 26315 hybrid_clock.cc:648] HybridClock initialized: now 1786084746311920 us; error 26 us; skew 500 ppm
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:06.314451 26315 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:39:06.314893 26315 webserver.cc:460] Webserver started at http://127.11.114.194:37057/ using document root <none> and password file <none>
I20260807 06:39:06.315143 26315 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:06.315212 26315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:06.315361 26315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:06.316392 26315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "6aa9d06acda053aff53631f6df65590b"
server_key_iv: "f1d646682b055681d8256dabc506a979"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.316764 26315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "6aa9d06acda053aff53631f6df65590b"
server_key_iv: "f1d646682b055681d8256dabc506a979"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.318197 26315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 06:39:06.318986 26332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.319190 26315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:39:06.319288 26315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "6aa9d06acda053aff53631f6df65590b"
server_key_iv: "f1d646682b055681d8256dabc506a979"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.319368 26315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:06.328577 26315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:06.330955 26315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:06.331121 26315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:06.331370 26315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:06.331692 26315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:06.331754 26315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.331815 26315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:06.331842 26315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.338018 26315 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:45993
I20260807 06:39:06.338095 26486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:45993 every 8 connection(s)
I20260807 06:39:06.338426 26315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:39:06.342541 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 26315
I20260807 06:39:06.342666 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:39:06.342862 11723 external_mini_cluster.cc:1428] Setting key 4083fa40e78a7985df1c1bdcf54f7321
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:39:06.349340 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.339245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:52483 (local address 127.11.114.254:46565)
0807 06:39:06.339334 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.339336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.339536 (+ 200us) server_negotiation.cc:407] Connection header received
0807 06:39:06.340074 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.340076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.340090 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.340141 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:06.341100 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.341713 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.342820 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.343089 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.343165 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.345707 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:06.345709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:06.345710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:06.345720 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:06.347549 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.348094 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.348096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.348097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.348164 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.348492 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.348494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.348494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.348675 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:06.348745 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.349153 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.349248 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:39:06.349553 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.339206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46565 (local address 127.11.114.194:52483)
0807 06:39:06.339396 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.339443 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:06.339958 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:06.340209 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:06.340213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:06.340795 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:06.340986 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.340990 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.341848 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.341849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.342695 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.342698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.343171 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.343173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.343317 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.343902 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:06.343913 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:06.345602 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:06.347653 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:06.347659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:06.347662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:06.347998 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:06.348265 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:06.348266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:06.348266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:06.348415 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:06.348767 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:06.348774 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:06.349007 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 06:39:06.349118 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 06:39:06.349294 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
I20260807 06:39:06.350140 26488 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46565
I20260807 06:39:06.350237 26488 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:06.350445 26488 heartbeater.cc:511] Master 127.11.114.254:46565 requested a full tablet report, sending...
I20260807 06:39:06.350878 26057 ts_manager.cc:194] Registered new tserver with Master: 4aed2bcab9794d1aa8fa3e59c0cdd822 (127.11.114.194:45993)
I20260807 06:39:06.351219 26057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:52483
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:39:06.372263 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46565
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:39:06.455333 26494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:06.455569 26494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:06.455606 26494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:06.456728 26494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:06.456796 26494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:06.456842 26494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:39:06.458345 26494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:06.458619 26494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:06.458858 26494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:06.464411 26501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:06.464448 26506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:06.464422 26502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:06.464510 26494 server_base.cc:1034] running on GCE node
I20260807 06:39:06.464830 26494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:06.465051 26494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:06.466190 26494 hybrid_clock.cc:648] HybridClock initialized: now 1786084746466185 us; error 41 us; skew 500 ppm
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:06.468909 26494 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:39:06.469447 26494 webserver.cc:460] Webserver started at http://127.11.114.195:36139/ using document root <none> and password file <none>
I20260807 06:39:06.469744 26494 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:06.469812 26494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:06.469945 26494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:39:06.471064 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "9f9370d2a5724862a1963a2c0d987c1c"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "858e46cea99023068766de230e490127"
server_key_iv: "b3afff1fc89cba18fa62897bde741f5d"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.471419 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "9f9370d2a5724862a1963a2c0d987c1c"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "858e46cea99023068766de230e490127"
server_key_iv: "b3afff1fc89cba18fa62897bde741f5d"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.472885 26494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:39:06.473980 26518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.474248 26494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:39:06.474344 26494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "9f9370d2a5724862a1963a2c0d987c1c"
format_stamp: "Formatted at 2026-08-07 06:39:06 on dist-test-slave-gkw7"
server_key: "858e46cea99023068766de230e490127"
server_key_iv: "b3afff1fc89cba18fa62897bde741f5d"
server_key_version: "encryptionkey@0"
I20260807 06:39:06.474424 26494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:06.490125 26494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:06.492177 26494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:06.492363 26494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:06.492617 26494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:06.492949 26494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:06.493005 26494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.493040 26494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:06.493083 26494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:06.499534 26494 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:39213
I20260807 06:39:06.499574 26668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:39213 every 8 connection(s)
I20260807 06:39:06.499917 26494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:39:06.507581 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 26494
I20260807 06:39:06.507696 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:39:06.507882 11723 external_mini_cluster.cc:1428] Setting key afa46ce483ba092cad4cf40924632b0d
I20260807 06:39:06.510360 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.500719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:41209 (local address 127.11.114.254:46565)
0807 06:39:06.500771 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.500774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.500961 (+ 187us) server_negotiation.cc:407] Connection header received
0807 06:39:06.501501 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.501503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.501517 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.501595 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:06.502558 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.503075 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.504175 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.504537 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.504622 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.506800 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:06.506801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:06.506802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:06.506809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:06.508643 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.509145 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.509146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.509147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.509227 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.509576 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.509577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.509578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.509728 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:06.509825 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.510137 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.510254 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:39:06.510547 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.500685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46565 (local address 127.11.114.195:41209)
0807 06:39:06.500833 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.500878 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:06.501386 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:06.501666 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:06.501670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:06.502266 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:06.502458 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.502463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.503173 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.503174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.504080 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.504082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.504632 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.504633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.504771 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.505244 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:06.505253 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:06.506686 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:06.508754 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:06.508762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:06.508764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:06.509028 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:06.509322 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:06.509323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:06.509324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:06.509450 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:06.509812 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:06.509820 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:06.509971 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 06:39:06.510092 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 06:39:06.510274 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260807 06:39:06.511097 26669 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46565
I20260807 06:39:06.511207 26669 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:06.511390 26669 heartbeater.cc:511] Master 127.11.114.254:46565 requested a full tablet report, sending...
I20260807 06:39:06.511787 26057 ts_manager.cc:194] Registered new tserver with Master: 9f9370d2a5724862a1963a2c0d987c1c (127.11.114.195:39213)
I20260807 06:39:06.512226 26057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:41209
I20260807 06:39:06.519783 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:39:06.527652 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.521222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.11.114.254:46565)
0807 06:39:06.521267 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.521268 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.521400 (+ 132us) server_negotiation.cc:407] Connection header received
0807 06:39:06.521444 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.521445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.521457 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.521513 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:06.522135 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.522689 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.523484 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.523726 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.523794 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.524905 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:06.524906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:06.524907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:06.524913 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:06.526605 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.526908 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.526909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.526909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.526964 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:06.527166 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:06.527167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:06.527167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:06.527342 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:06.527433 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.527521 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.527590 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:39:06.529340 26057 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:39:06.530906 26057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:39:06.537111 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.533353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:45993 (local address 127.0.0.1:56694)
0807 06:39:06.533597 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.533607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:06.533650 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:06.534158 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:06.534160 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:06.534162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:06.534202 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.534203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.535053 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.535054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.536033 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.536036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.536820 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.536821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.536900 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.536914 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 06:39:06.536988 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:39:06.537033 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260807 06:39:06.537147 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.533505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.11.114.194:45993)
0807 06:39:06.533984 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.533989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.533999 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:39:06.534040 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.534042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.534057 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.534162 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:06.534268 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.534948 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.536103 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.536735 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.536810 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.536862 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.537015 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.537056 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":115,"threads_started":1}
I20260807 06:39:06.537613 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.533172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:39213 (local address 127.0.0.1:33124)
0807 06:39:06.533284 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.533328 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:06.533427 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:06.533861 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:06.533864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:06.533867 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:06.533960 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.533962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.534932 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.534933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.536321 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.536324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.537316 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.537317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.537397 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.537401 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:39:06.537462 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 06:39:06.537500 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":54,"thread_start_us":38,"threads_started":1}
I20260807 06:39:06.537621 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.533324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33124 (local address 127.11.114.195:39213)
0807 06:39:06.533657 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.533663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.533675 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:39:06.533717 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.533719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.533735 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.533839 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:06.534058 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.534832 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.536419 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.537156 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.537230 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.537281 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.537478 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.537521 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":103,"threads_started":1}
I20260807 06:39:06.537962 26395 tablet_service.cc:1467] Processing CreateTablet for tablet 4f369902f498491aa07f6ce4ee538165 (DEFAULT_TABLE table=test-table [id=9ab7e0ada30c4d419f27f337e37ba979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:39:06.538317 26395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f369902f498491aa07f6ce4ee538165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:06.538949 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.533637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43665 (local address 127.0.0.1:54614)
0807 06:39:06.533824 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.533834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:06.533878 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:06.535190 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:06.535191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:06.535193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:06.535287 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.535288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.536524 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.536525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.537662 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.537665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.538731 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.538732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.538790 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.538794 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:39:06.538846 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:39:06.538884 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":41,"thread_start_us":88,"threads_started":1}
I20260807 06:39:06.538975 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.533899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.11.114.193:43665)
0807 06:39:06.535019 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.535023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.535031 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:39:06.535085 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.535087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.535102 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.535178 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:06.535365 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.536418 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.537982 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.538622 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.538746 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.538803 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.538858 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.538897 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":100,"threads_started":1}
I20260807 06:39:06.539701 26220 tablet_service.cc:1467] Processing CreateTablet for tablet 4f369902f498491aa07f6ce4ee538165 (DEFAULT_TABLE table=test-table [id=9ab7e0ada30c4d419f27f337e37ba979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:39:06.540060 26220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f369902f498491aa07f6ce4ee538165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:06.542287 26583 tablet_service.cc:1467] Processing CreateTablet for tablet 4f369902f498491aa07f6ce4ee538165 (DEFAULT_TABLE table=test-table [id=9ab7e0ada30c4d419f27f337e37ba979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:39:06.542615 26583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f369902f498491aa07f6ce4ee538165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:06.543494 26696 tablet_bootstrap.cc:492] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be: Bootstrap starting.
I20260807 06:39:06.544096 26696 tablet_bootstrap.cc:654] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:06.544569 26696 log.cc:824] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:06.545435 26696 tablet_bootstrap.cc:492] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be: No bootstrap required, opened a new log
I20260807 06:39:06.545508 26696 ts_tablet_manager.cc:1397] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:39:06.545712 26698 tablet_bootstrap.cc:492] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: Bootstrap starting.
I20260807 06:39:06.546222 26698 tablet_bootstrap.cc:654] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:06.546655 26698 log.cc:824] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:06.546913 26695 tablet_bootstrap.cc:492] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: Bootstrap starting.
I20260807 06:39:06.547279 26696 raft_consensus.cc:348] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.547428 26696 raft_consensus.cc:374] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:06.547470 26696 raft_consensus.cc:729] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817dccc2978c4f49ac28752d4233f3be, State: Initialized, Role: FOLLOWER
I20260807 06:39:06.547488 26698 tablet_bootstrap.cc:492] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: No bootstrap required, opened a new log
I20260807 06:39:06.547509 26695 tablet_bootstrap.cc:654] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:06.547549 26698 ts_tablet_manager.cc:1397] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:39:06.547623 26696 consensus_queue.cc:260] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.547930 26308 heartbeater.cc:503] Master 127.11.114.254:46565 was elected leader, sending a full tablet report...
I20260807 06:39:06.548053 26695 log.cc:824] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:06.548233 26696 ts_tablet_manager.cc:1428] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:39:06.548889 26695 tablet_bootstrap.cc:492] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: No bootstrap required, opened a new log
I20260807 06:39:06.548964 26695 ts_tablet_manager.cc:1397] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:39:06.549625 26698 raft_consensus.cc:348] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.549782 26698 raft_consensus.cc:374] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:06.549816 26698 raft_consensus.cc:729] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f9370d2a5724862a1963a2c0d987c1c, State: Initialized, Role: FOLLOWER
I20260807 06:39:06.549924 26698 consensus_queue.cc:260] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.550185 26698 ts_tablet_manager.cc:1428] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:39:06.550218 26669 heartbeater.cc:503] Master 127.11.114.254:46565 was elected leader, sending a full tablet report...
I20260807 06:39:06.551016 26695 raft_consensus.cc:348] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.551162 26695 raft_consensus.cc:374] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:06.551223 26695 raft_consensus.cc:729] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aed2bcab9794d1aa8fa3e59c0cdd822, State: Initialized, Role: FOLLOWER
I20260807 06:39:06.551334 26695 consensus_queue.cc:260] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.551537 26695 ts_tablet_manager.cc:1428] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:39:06.551596 26488 heartbeater.cc:503] Master 127.11.114.254:46565 was elected leader, sending a full tablet report...
I20260807 06:39:06.553651 26705 raft_consensus.cc:482] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:39:06.553723 26705 raft_consensus.cc:504] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.554028 26705 leader_election.cc:288] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f9370d2a5724862a1963a2c0d987c1c (127.11.114.195:39213), 817dccc2978c4f49ac28752d4233f3be (127.11.114.193:43665)
I20260807 06:39:06.557184 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.554273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:51067 (local address 127.11.114.195:39213)
0807 06:39:06.554315 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.554317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.554325 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:39:06.554368 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.554369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.554378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.554429 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:06.554742 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.555478 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.556442 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.556827 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.556881 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.556888 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.557067 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.557112 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:39:06.557206 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.554369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:42603 (local address 127.11.114.193:43665)
0807 06:39:06.554410 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.554412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.554564 (+ 152us) server_negotiation.cc:407] Connection header received
0807 06:39:06.554593 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.554594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.554604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.554668 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:06.554821 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.555503 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.556457 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.556877 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.556946 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.556954 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.557107 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.557143 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:39:06.557219 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.554195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:39213 (local address 127.11.114.194:51067)
0807 06:39:06.554245 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.554259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:06.554309 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:06.554478 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:06.554480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:06.554482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:06.554569 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.554571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.555568 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.555569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.556350 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.556352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.556905 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.556906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.556987 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.556990 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:39:06.557042 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:39:06.557123 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":30}
I20260807 06:39:06.557289 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.554293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43665 (local address 127.11.114.194:42603)
0807 06:39:06.554467 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.554476 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:06.554527 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:06.554676 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:06.554677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:06.554678 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:06.554737 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.554739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.555599 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.555600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.556357 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:06.556359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.556960 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.556961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:06.557013 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.557016 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:39:06.557090 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:39:06.557126 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":20,"thread_start_us":87,"threads_started":1}
I20260807 06:39:06.557566 26609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f369902f498491aa07f6ce4ee538165" candidate_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f9370d2a5724862a1963a2c0d987c1c" is_pre_election: true
I20260807 06:39:06.557703 26609 raft_consensus.cc:2454] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aed2bcab9794d1aa8fa3e59c0cdd822 in term 0.
I20260807 06:39:06.557709 26247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f369902f498491aa07f6ce4ee538165" candidate_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817dccc2978c4f49ac28752d4233f3be" is_pre_election: true
I20260807 06:39:06.557842 26247 raft_consensus.cc:2454] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aed2bcab9794d1aa8fa3e59c0cdd822 in term 0.
I20260807 06:39:06.557897 26337 leader_election.cc:302] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aed2bcab9794d1aa8fa3e59c0cdd822, 9f9370d2a5724862a1963a2c0d987c1c; no voters:
I20260807 06:39:06.558008 26705 raft_consensus.cc:2781] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:39:06.558072 26705 raft_consensus.cc:482] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:39:06.558089 26705 raft_consensus.cc:3037] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:06.558759 26705 raft_consensus.cc:504] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.558880 26705 leader_election.cc:288] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [CANDIDATE]: Term 1 election: Requested vote from peers 9f9370d2a5724862a1963a2c0d987c1c (127.11.114.195:39213), 817dccc2978c4f49ac28752d4233f3be (127.11.114.193:43665)
I20260807 06:39:06.559000 26609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f369902f498491aa07f6ce4ee538165" candidate_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f9370d2a5724862a1963a2c0d987c1c"
I20260807 06:39:06.559034 26247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f369902f498491aa07f6ce4ee538165" candidate_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817dccc2978c4f49ac28752d4233f3be"
I20260807 06:39:06.559077 26609 raft_consensus.cc:3037] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:06.559116 26247 raft_consensus.cc:3037] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:06.559724 26609 raft_consensus.cc:2454] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aed2bcab9794d1aa8fa3e59c0cdd822 in term 1.
I20260807 06:39:06.559854 26247 raft_consensus.cc:2454] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aed2bcab9794d1aa8fa3e59c0cdd822 in term 1.
I20260807 06:39:06.559887 26337 leader_election.cc:302] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aed2bcab9794d1aa8fa3e59c0cdd822, 9f9370d2a5724862a1963a2c0d987c1c; no voters:
I20260807 06:39:06.559957 26705 raft_consensus.cc:2781] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:39:06.560098 26705 raft_consensus.cc:686] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 1 LEADER]: Becoming Leader. State: Replica: 4aed2bcab9794d1aa8fa3e59c0cdd822, State: Running, Role: LEADER
I20260807 06:39:06.560205 26705 consensus_queue.cc:237] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } }
I20260807 06:39:06.560926 26057 catalog_manager.cc:5495] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4aed2bcab9794d1aa8fa3e59c0cdd822 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aed2bcab9794d1aa8fa3e59c0cdd822" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 45993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 } health_report { overall_health: UNKNOWN } } }
I20260807 06:39:06.572975 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.570753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56700 (local address 127.11.114.194:45993)
0807 06:39:06.570795 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:06.570797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:06.570806 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:39:06.570858 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:06.570858 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:06.570873 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:06.570932 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:39:06.571016 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.571480 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.572231 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:06.572410 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:06.572462 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:06.572587 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:39:06.572680 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:39:06.572744 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:06.572884 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 06:39:06.572903 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:39:06.574824 26247 raft_consensus.cc:1261] T 4f369902f498491aa07f6ce4ee538165 P 817dccc2978c4f49ac28752d4233f3be [term 1 FOLLOWER]: Refusing update from remote peer 4aed2bcab9794d1aa8fa3e59c0cdd822: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:39:06.574852 26609 raft_consensus.cc:1261] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 1 FOLLOWER]: Refusing update from remote peer 4aed2bcab9794d1aa8fa3e59c0cdd822: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:39:06.575151 26705 consensus_queue.cc:1035] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "817dccc2978c4f49ac28752d4233f3be" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:39:06.575305 26707 consensus_queue.cc:1035] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f9370d2a5724862a1963a2c0d987c1c" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:39:06.578004 26710 mvcc.cc:205] Tried to move back new op lower bound from 7315803121967947776 to 7315803121911189504. Current Snapshot: MvccSnapshot[applied={T|T < 7315803121967947776}]
I20260807 06:39:06.578510 26713 mvcc.cc:205] Tried to move back new op lower bound from 7315803121967947776 to 7315803121911189504. Current Snapshot: MvccSnapshot[applied={T|T < 7315803121967947776}]
I20260807 06:39:06.578708 26714 mvcc.cc:205] Tried to move back new op lower bound from 7315803121967947776 to 7315803121911189504. Current Snapshot: MvccSnapshot[applied={T|T < 7315803121967947776 or (T in {7315803121967947776})}]
I20260807 06:39:06.580847 26057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:39:06.580950 26057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:39:06.581712 26057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5920d15178644da2abdb4366509eb351: Sending DeleteTablet for 3 replicas of tablet 4f369902f498491aa07f6ce4ee538165
I20260807 06:39:06.582085 26583 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f369902f498491aa07f6ce4ee538165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:39:06 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:33124
I20260807 06:39:06.582218 26395 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f369902f498491aa07f6ce4ee538165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:39:06 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:56694
I20260807 06:39:06.582393 26724 tablet_replica.cc:330] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: stopping tablet replica
I20260807 06:39:06.582445 26220 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f369902f498491aa07f6ce4ee538165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:39:06 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:54614
I20260807 06:39:06.582593 26724 raft_consensus.cc:2229] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:39:06.582603 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 26138
I20260807 06:39:06.582664 26725 tablet_replica.cc:330] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: stopping tablet replica
I20260807 06:39:06.582760 26725 raft_consensus.cc:2229] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:39:06.582872 26725 raft_consensus.cc:2258] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:39:06.583015 26724 raft_consensus.cc:2258] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:39:06.583644 26725 ts_tablet_manager.cc:1905] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:39:06.585047 26725 ts_tablet_manager.cc:1918] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:39:06.585134 26725 log.cc:1196] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/4f369902f498491aa07f6ce4ee538165
I20260807 06:39:06.585407 26725 ts_tablet_manager.cc:1939] T 4f369902f498491aa07f6ce4ee538165 P 4aed2bcab9794d1aa8fa3e59c0cdd822: Deleting consensus metadata
I20260807 06:39:06.586092 26041 catalog_manager.cc:4841] TS 4aed2bcab9794d1aa8fa3e59c0cdd822 (127.11.114.194:45993): tablet 4f369902f498491aa07f6ce4ee538165 (table test-table [id=9ab7e0ada30c4d419f27f337e37ba979]) successfully deleted
I20260807 06:39:06.587065 26724 ts_tablet_manager.cc:1905] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:39:06.588315 26724 ts_tablet_manager.cc:1918] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:39:06.588402 26724 log.cc:1196] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/4f369902f498491aa07f6ce4ee538165
I20260807 06:39:06.588675 26724 ts_tablet_manager.cc:1939] T 4f369902f498491aa07f6ce4ee538165 P 9f9370d2a5724862a1963a2c0d987c1c: Deleting consensus metadata
I20260807 06:39:06.589056 26041 catalog_manager.cc:4841] TS 9f9370d2a5724862a1963a2c0d987c1c (127.11.114.195:39213): tablet 4f369902f498491aa07f6ce4ee538165 (table test-table [id=9ab7e0ada30c4d419f27f337e37ba979]) successfully deleted
W20260807 06:39:06.593173 26041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:43665 (error 108)
I20260807 06:39:06.593824 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:06.593662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43665 (local address 127.0.0.1:54622)
0807 06:39:06.593719 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:06.593760 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:43665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:39:06.593879 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 26315
W20260807 06:39:06.600241 26041 catalog_manager.cc:4584] TS 817dccc2978c4f49ac28752d4233f3be (127.11.114.193:43665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f369902f498491aa07f6ce4ee538165: Network error: Client connection negotiation failed: client connection to 127.11.114.193:43665: connect: Connection refused (error 111)
I20260807 06:39:06.604364 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 26494
I20260807 06:39:06.614068 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 26019
2026-08-07T06:39:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2188 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26730](info): set up 2 sockets
Aug 07 06:39:06 dist-test-slave-gkw7 krb5kdc[26730](info): commencing operation
krb5kdc: starting...
W20260807 06:39:07.852101 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:39:07 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:39:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:39:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.114.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.114.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/rangeradmin_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/rangeradmin_127.11.114.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.114.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.114.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/rangerlookup_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/rangerlookup_127.11.114.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.114.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.114.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/HTTP_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/HTTP_127.11.114.212@KRBTEST.COM.keytab.
I20260807 06:39:08.025076 11723 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZbZY4B/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/postgres -l logfile start
2026-08-07 06:39:10.650 UTC [26770] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 06:39:10.650 UTC [26770] LOG: listening on IPv4 address "127.11.114.212", port 44443
2026-08-07 06:39:10.654 UTC [26770] LOG: listening on Unix socket "/tmp/.s.PGSQL.44443"
2026-08-07 06:39:10.676 UTC [26774] LOG: database system was shut down at 2026-08-07 06:39:09 UTC
2026-08-07 06:39:10.680 UTC [26770] LOG: database system is ready to accept connections
I20260807 06:39:11.912020 11723 mini_postgres.cc:96] Postgres bound to 44443
2026-08-07 06:39:11.920 UTC [26785] FATAL: database "slave" does not exist
127.11.114.212:44443 - accepting connections
I20260807 06:39:11.921027 11723 mini_ranger.cc:161] Starting Ranger...
I20260807 06:39:11.937752 11723 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 06:39:12.439787 11723 mini_ranger.cc:87] Created ranger Postgres database
I20260807 06:39:12.439880 11723 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-admin
2026-08-07 06:39:12,769 [I] DB FLAVOR :POSTGRES
2026-08-07 06:39:12,770 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:39:12,770 [I] Checking connection..
2026-08-07 06:39:12,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 06:39:13,060 [I] Checking connection passed.
2026-08-07 06:39:13,060 [I] --------- Verifying version history table ---------
2026-08-07 06:39:13,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:39:13,317 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:39:13,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:39:13,574 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:39:13,575 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 06:39:13,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 06:39:13.820 UTC [26880] WARNING: there is no transaction in progress
2026-08-07 06:39:13,829 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 06:39:13,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:39:14,098 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 06:39:14,099 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:39:14,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 06:39:14,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 06:39:14,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-07 06:39:14,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:39:15,123 [I] Table x_portal_user does not exist in database ranger
2026-08-07 06:39:15,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:39:15,378 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 06:39:15,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:39:15,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:39:15,905 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 06:39:15,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 06:39:16.895 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:16.911 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:16.928 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:16.944 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:16.960 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:16.979 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.038 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.046 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.061 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.071 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.081 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.095 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.106 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.112 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.119 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17.126 UTC [27019] WARNING: there is no transaction in progress
2026-08-07 06:39:17,764 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 06:39:17,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 06:39:18,003 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 06:39:18,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:39:18,255 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 06:39:18,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:39:18,519 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 06:39:18,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:39:18,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:39:19,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-07 06:39:19,294 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 06:39:19,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 06:39:19,559 [I] DB_PATCHES have already been applied
I20260807 06:39:19.565465 11723 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZbZY4B/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 06:39:19.565578 11723 mini_ranger.cc:193] Using host: 127.11.114.212
I20260807 06:39:19.569078 11723 mini_ranger.cc:227] Ranger admin URL: http://127.11.114.212:34977
Aug 07, 2026 6:39:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:39:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 06:39:20.853331 11723 mini_ranger.cc:160] Time spent starting Ranger: real 8.932s user 0.000s sys 0.011s
Aug 07, 2026 6:39:40 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:39:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:39:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:39:40 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:39:42 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 06:39:42.616010 11723 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.114.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.114.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/rangerkms_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/rangerkms_127.11.114.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.114.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.114.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.114.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/HTTP_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.114.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/HTTP_127.11.114.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 06:39:42 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084782, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 06:39:42.714407 11723 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 06:39:42.726254 11723 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 06:39:43.637239 11723 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 06:39:43.637339 11723 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms
I20260807 06:39:43.637367 11723 mini_ranger_kms.cc:227] Using postgres at 127.11.114.212:44443
2026-08-07 06:39:44,014 [I] DB FLAVOR :POSTGRES
2026-08-07 06:39:44,014 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:39:44,014 [I] Checking connection
2026-08-07 06:39:44,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 06:39:44,274 [I] connection success
2026-08-07 06:39:44,274 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 06:39:44,275 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 06:39:44,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 06:39:44,524 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 06:39:44,524 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:39:44,525 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 06:39:44,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:44443/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 06:39:44,810 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 06:39:45.283499 11723 mini_ranger_kms.cc:310] Created kms service
I20260807 06:39:45.482935 11723 mini_ranger_kms.cc:326] Created kudu user
I20260807 06:39:45.527083 11723 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 06:39:45 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084782, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:39:45.919667 11723 mini_ranger_kms.cc:384] Added ranger policy
I20260807 06:39:45.919790 11723 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 06:39:45.919811 11723 mini_ranger_kms.cc:240] Using host: 127.11.114.212
I20260807 06:39:45.923878 11723 mini_ranger_kms.cc:276] Ranger KMS PID: 27267
I20260807 06:39:45.924005 11723 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.114.212:37361
Aug 07, 2026 6:39:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:39:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:39:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:39:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:39:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:39:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 06:39:48 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084788, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.114.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:48.614367 11723 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.900s user 0.000s sys 0.354s
I20260807 06:39:48.614491 11723 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 06:39:48.614579 11723 mini_ranger_kms.cc:401] 127.11.114.212:37361/kms/v1/keys
Aug 07 06:39:48 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084788, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.114.212@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
Aug 07 06:39:49 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084788, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.114.212@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
Aug 07 06:39:51 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:39:51.348520 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:36459
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:36459
--ranger_config_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:37361/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:39:51.434203 27340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:51.434455 27340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:51.434494 27340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:51.435662 27340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:39:51.435727 27340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:51.435753 27340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:51.435773 27340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:39:51.435806 27340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:39:51.437486 27340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.114.212:37361/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:36459
--ranger_config_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:36459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:51.437834 27340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:51.438071 27340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:51.444615 27346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:51.444640 27345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:51.444617 27349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:51.444854 27340 server_base.cc:1034] running on GCE node
I20260807 06:39:51.445114 27340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:51.445469 27340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:51.446637 27340 hybrid_clock.cc:648] HybridClock initialized: now 1786084791446614 us; error 41 us; skew 500 ppm
Aug 07 06:39:51 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:51.449482 27340 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:39:51.450035 27340 webserver.cc:460] Webserver started at http://127.11.114.254:37767/ using document root <none> and password file <none>
I20260807 06:39:51.450314 27340 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:51.450377 27340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:51.450537 27340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:39:51 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:39:51.496762 27340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "e02d9e21667b4f9ebd1f6abf70377540"
format_stamp: "Formatted at 2026-08-07 06:39:51 on dist-test-slave-gkw7"
server_key: "e57387ef88e386720e2cf89c5c2c9ae5"
server_key_iv: "03ab2096523c41b7c5a2c7a857d8547f"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:51.497211 27340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "e02d9e21667b4f9ebd1f6abf70377540"
format_stamp: "Formatted at 2026-08-07 06:39:51 on dist-test-slave-gkw7"
server_key: "e57387ef88e386720e2cf89c5c2c9ae5"
server_key_iv: "03ab2096523c41b7c5a2c7a857d8547f"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:51.499096 27340 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 06:39:51.514112 27359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:51.514317 27340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:39:51.514426 27340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "e02d9e21667b4f9ebd1f6abf70377540"
format_stamp: "Formatted at 2026-08-07 06:39:51 on dist-test-slave-gkw7"
server_key: "e57387ef88e386720e2cf89c5c2c9ae5"
server_key_iv: "03ab2096523c41b7c5a2c7a857d8547f"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:51.514534 27340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:51.536885 27340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:51.539011 27340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:51.539203 27340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:51.544674 27340 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:36459
I20260807 06:39:51.544790 27431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:36459 every 8 connection(s)
I20260807 06:39:51.545132 27340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:39:51.545570 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 27340
I20260807 06:39:51.545699 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:39:51.545928 11723 external_mini_cluster.cc:1428] Setting key cf59adc5a2c9ac582406d2b67606b0cf
I20260807 06:39:51.546183 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:51.549813 27433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540: Bootstrap starting.
I20260807 06:39:51.550570 27433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:51.550997 27433 log.cc:824] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:51.551962 27433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540: No bootstrap required, opened a new log
Aug 07 06:39:51 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:39:51.554236 27433 raft_consensus.cc:348] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02d9e21667b4f9ebd1f6abf70377540" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36459 } }
I20260807 06:39:51.554411 27433 raft_consensus.cc:374] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:51.554494 27433 raft_consensus.cc:729] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02d9e21667b4f9ebd1f6abf70377540, State: Initialized, Role: FOLLOWER
I20260807 06:39:51.554654 27433 consensus_queue.cc:260] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02d9e21667b4f9ebd1f6abf70377540" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36459 } }
I20260807 06:39:51.554762 27433 raft_consensus.cc:388] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:39:51.554836 27433 raft_consensus.cc:482] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:39:51.554898 27433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:51.555684 27433 raft_consensus.cc:504] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02d9e21667b4f9ebd1f6abf70377540" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36459 } }
I20260807 06:39:51.555804 27433 leader_election.cc:302] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e02d9e21667b4f9ebd1f6abf70377540; no voters:
I20260807 06:39:51.555995 27433 leader_election.cc:288] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:39:51.556067 27438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:39:51.556385 27438 raft_consensus.cc:686] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [term 1 LEADER]: Becoming Leader. State: Replica: e02d9e21667b4f9ebd1f6abf70377540, State: Running, Role: LEADER
I20260807 06:39:51.556409 27433 sys_catalog.cc:564] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:39:51.557157 27438 consensus_queue.cc:237] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02d9e21667b4f9ebd1f6abf70377540" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36459 } }
I20260807 06:39:51.557992 27440 sys_catalog.cc:455] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e02d9e21667b4f9ebd1f6abf70377540. Latest consensus state: current_term: 1 leader_uuid: "e02d9e21667b4f9ebd1f6abf70377540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02d9e21667b4f9ebd1f6abf70377540" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36459 } } }
I20260807 06:39:51.558072 27440 sys_catalog.cc:458] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [sys.catalog]: This master's current role is: LEADER
I20260807 06:39:51.558336 27433 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 06:39:51.558481 27453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:39:51.558346 27439 sys_catalog.cc:455] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e02d9e21667b4f9ebd1f6abf70377540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02d9e21667b4f9ebd1f6abf70377540" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 36459 } } }
I20260807 06:39:51.558915 27439 sys_catalog.cc:458] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540 [sys.catalog]: This master's current role is: LEADER
I20260807 06:39:51.559088 27453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:39:51.559283 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:51.546476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44350 (local address 127.11.114.254:36459)
0807 06:39:51.546745 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:51.546756 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:51.546843 (+ 87us) server_negotiation.cc:407] Connection header received
0807 06:39:51.547605 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:51.547616 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:51.547685 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:51.547835 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:51.548633 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:51.550137 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:51.551294 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:51.551788 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:51.551902 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:51.554840 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:51.554844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:51.554845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:51.554858 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:51.557118 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:51.557564 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:51.557567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:51.557567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:51.557663 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:51.557900 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:51.557901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:51.557901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:51.558246 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:51.558397 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:51.558928 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 06:39:51.559064 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":140,"threads_started":1}
I20260807 06:39:51.567625 27453 catalog_manager.cc:1347] Generated new cluster ID: 4a7880614b5b46f7853789ffc0db4972
I20260807 06:39:51.567742 27453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:39:51.580411 27453 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:39:51.581246 27453 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:39:51.586805 27453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540: Generated new TSK 0
I20260807 06:39:51.587061 27453 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:39:52.296445 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:37361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36459
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:39:52.417145 27488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:52.417383 27488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:52.417428 27488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:52.418699 27488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:52.418816 27488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:52.418911 27488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:39:52.421097 27488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.114.212:37361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:52.421365 27488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:52.421698 27488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:52.428457 27499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:52.428540 27498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:52.428467 27503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:52.428709 27488 server_base.cc:1034] running on GCE node
I20260807 06:39:52.428969 27488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:52.429225 27488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:52.430392 27488 hybrid_clock.cc:648] HybridClock initialized: now 1786084792430379 us; error 33 us; skew 500 ppm
Aug 07 06:39:52 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:52.433159 27488 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:39:52.433754 27488 webserver.cc:460] Webserver started at http://127.11.114.193:39893/ using document root <none> and password file <none>
I20260807 06:39:52.434029 27488 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:52.434077 27488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:52.434218 27488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:39:52 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:39:52.448835 27488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "34e9455aee984504b830dfb178e90392"
format_stamp: "Formatted at 2026-08-07 06:39:52 on dist-test-slave-gkw7"
server_key: "47b32bc358587fe2c932ce16fe877458"
server_key_iv: "6a6c69bfc59f184f56cb215ad1cae088"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:52.449285 27488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "34e9455aee984504b830dfb178e90392"
format_stamp: "Formatted at 2026-08-07 06:39:52 on dist-test-slave-gkw7"
server_key: "47b32bc358587fe2c932ce16fe877458"
server_key_iv: "6a6c69bfc59f184f56cb215ad1cae088"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:52.451689 27488 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 06:39:52.473687 27516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:52.474017 27488 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260807 06:39:52.474172 27488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "34e9455aee984504b830dfb178e90392"
format_stamp: "Formatted at 2026-08-07 06:39:52 on dist-test-slave-gkw7"
server_key: "47b32bc358587fe2c932ce16fe877458"
server_key_iv: "6a6c69bfc59f184f56cb215ad1cae088"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:52.474483 27488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:52.491529 27488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:52.494282 27488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:52.494506 27488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:52.494803 27488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:52.495254 27488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:52.495347 27488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:52.495447 27488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:52.495522 27488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:52.504995 27488 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:39155
I20260807 06:39:52.505111 27660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:39155 every 8 connection(s)
I20260807 06:39:52.505508 27488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:39:52 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:39:52.514183 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 27488
I20260807 06:39:52.514298 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:39:52.514510 11723 external_mini_cluster.cc:1428] Setting key 6d9901e9727255c8e318e43cd4ad5e72
I20260807 06:39:52.520699 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:52.506480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:36459 (local address 127.11.114.193:46071)
0807 06:39:52.506833 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:52.506884 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:52.507570 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:52.508065 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:52.508070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:52.508329 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:52.508619 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:52.508626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:52.509767 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:52.509768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:52.511091 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:52.511094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:52.511635 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:52.511636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:52.511796 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:52.512479 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:52.512491 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:52.514398 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:52.518708 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:52.518717 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:52.518720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:52.519206 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:52.519441 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:52.519442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:52.519443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:52.519635 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:52.519974 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:52.519981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:52.520148 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 06:39:52.520246 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:39:52.520418 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":108,"threads_started":1}
I20260807 06:39:52.521235 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:52.506562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:46071 (local address 127.11.114.254:36459)
0807 06:39:52.507803 (+ 1241us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:52.507805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:52.507814 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:39:52.507841 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:52.507842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:52.507854 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:52.507935 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:52.508748 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:52.509654 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:52.511200 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:52.511524 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:52.511615 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:52.514506 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:52.514507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:52.514508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:52.514516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:52.518596 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:52.519296 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:52.519297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:52.519297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:52.519369 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:52.519737 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:52.519737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:52.519738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:52.519890 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:52.520970 (+ 1080us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:52.521018 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 06:39:52.521111 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1208,"thread_start_us":96,"threads_started":1}
I20260807 06:39:52.523065 27663 heartbeater.cc:347] Connected to a master server at 127.11.114.254:36459
I20260807 06:39:52.523172 27663 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:52.523373 27663 heartbeater.cc:511] Master 127.11.114.254:36459 requested a full tablet report, sending...
I20260807 06:39:52.524118 27381 ts_manager.cc:194] Registered new tserver with Master: 34e9455aee984504b830dfb178e90392 (127.11.114.193:39155)
I20260807 06:39:52.525045 27381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:46071
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:39:52.577877 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:37361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36459
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:39:52 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 06:39:52.705770 27675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:52.706144 27675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:52.706234 27675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:52.707993 27675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:52.708245 27675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:52.708351 27675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:39:52.710542 27675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.114.212:37361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:52.711058 27675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:52.711400 27675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:52.718503 27682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:52.719126 27675 server_base.cc:1034] running on GCE node
W20260807 06:39:52.719308 27683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:52.719169 27685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:52.725915 27675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:52.726257 27675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:52.733624 27675 hybrid_clock.cc:648] HybridClock initialized: now 1786084792733596 us; error 53 us; skew 500 ppm
Aug 07 06:39:52 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:52.737454 27675 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:39:52.738159 27675 webserver.cc:460] Webserver started at http://127.11.114.194:43743/ using document root <none> and password file <none>
I20260807 06:39:52.738520 27675 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:52.738649 27675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:52.738883 27675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:39:52 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:39:52.756796 27675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "fce2a136a0ca4bd08bbfa09ba2b34033"
format_stamp: "Formatted at 2026-08-07 06:39:52 on dist-test-slave-gkw7"
server_key: "f9799beefa8b90c7bbf768e1ddb95314"
server_key_iv: "9ef565c54a6f540f96d98e2455217ae3"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:52.757593 27675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "fce2a136a0ca4bd08bbfa09ba2b34033"
format_stamp: "Formatted at 2026-08-07 06:39:52 on dist-test-slave-gkw7"
server_key: "f9799beefa8b90c7bbf768e1ddb95314"
server_key_iv: "9ef565c54a6f540f96d98e2455217ae3"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:52.759949 27675 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 06:39:52.777266 27695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:52.777559 27675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 06:39:52.777659 27675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "fce2a136a0ca4bd08bbfa09ba2b34033"
format_stamp: "Formatted at 2026-08-07 06:39:52 on dist-test-slave-gkw7"
server_key: "f9799beefa8b90c7bbf768e1ddb95314"
server_key_iv: "9ef565c54a6f540f96d98e2455217ae3"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:52.777777 27675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:52.801627 27675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:52.804141 27675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:52.804350 27675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:52.804605 27675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:52.805030 27675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:52.805117 27675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:52.805191 27675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:52.805248 27675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:52.812718 27675 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:43927
I20260807 06:39:52.813242 27675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:39:52.814347 27858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:43927 every 8 connection(s)
I20260807 06:39:52.818334 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 27675
I20260807 06:39:52.818456 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:39:52.818655 11723 external_mini_cluster.cc:1428] Setting key d353b1c4d0a1baed91dd42cbf793793e
Aug 07 06:39:52 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
I20260807 06:39:52.834650 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:52.815084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:36459 (local address 127.11.114.194:57119)
0807 06:39:52.815340 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:52.815382 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:52.816004 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:52.816353 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:52.816358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:52.816614 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:52.816896 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:52.816901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:52.817952 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:52.817953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:52.819340 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:52.819343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:52.821826 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:52.821827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:52.821988 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:52.822625 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:52.822636 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:52.824303 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:52.830970 (+ 6667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:52.830985 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:52.830989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:52.831339 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:52.832632 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:52.832634 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:52.832634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:52.832828 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:52.833897 (+ 1069us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:52.833906 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:52.834100 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 06:39:52.834208 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 06:39:52.834394 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":91,"threads_started":1}
I20260807 06:39:52.835290 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:52.815130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:57119 (local address 127.11.114.254:36459)
0807 06:39:52.815230 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:52.815232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:52.815502 (+ 270us) server_negotiation.cc:407] Connection header received
0807 06:39:52.816155 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:52.816156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:52.816175 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:52.816258 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:52.817014 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:52.817850 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:52.821381 (+ 3531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:52.821720 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:52.821817 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:52.828845 (+ 7028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:52.828847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:52.828847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:52.828859 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:52.830842 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:52.832471 (+ 1629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:52.832472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:52.832473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:52.832542 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:52.833630 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:52.833632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:52.833632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:52.833808 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:52.835058 (+ 1250us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:52.835102 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 06:39:52.835200 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
I20260807 06:39:52.837587 27859 heartbeater.cc:347] Connected to a master server at 127.11.114.254:36459
I20260807 06:39:52.837752 27859 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:52.838033 27859 heartbeater.cc:511] Master 127.11.114.254:36459 requested a full tablet report, sending...
I20260807 06:39:52.838497 27381 ts_manager.cc:194] Registered new tserver with Master: fce2a136a0ca4bd08bbfa09ba2b34033 (127.11.114.194:43927)
I20260807 06:39:52.839004 27381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:57119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:39:52.868211 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:37361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36459
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:39:52.999564 27865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:39:52.999783 27865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:39:52.999809 27865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:39:53.001029 27865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:39:53.001077 27865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:39:53.001104 27865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:39:53.002651 27865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.114.212:37361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:36459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:39:53.002893 27865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:39:53.003147 27865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:39:53.008643 27874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:53.008647 27872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:39:53.008750 27870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:39:53.009249 27865 server_base.cc:1034] running on GCE node
I20260807 06:39:53.009495 27865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:39:53.009917 27865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:39:53.011116 27865 hybrid_clock.cc:648] HybridClock initialized: now 1786084793011093 us; error 38 us; skew 500 ppm
Aug 07 06:39:53 dist-test-slave-gkw7 krb5kdc[26730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:39:53.014626 27865 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:39:53.015148 27865 webserver.cc:460] Webserver started at http://127.11.114.195:34745/ using document root <none> and password file <none>
I20260807 06:39:53.015393 27865 fs_manager.cc:359] Metadata directory not provided
I20260807 06:39:53.015440 27865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:39:53.015564 27865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:39:53 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:39:53.031371 27865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "a010edb5de824625b20f44b4a783c467"
format_stamp: "Formatted at 2026-08-07 06:39:53 on dist-test-slave-gkw7"
server_key: "d05db343a7fa79d760f23373de006139"
server_key_iv: "6a289082ad8cd09322a3f728d32e476d"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:53.031965 27865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "a010edb5de824625b20f44b4a783c467"
format_stamp: "Formatted at 2026-08-07 06:39:53 on dist-test-slave-gkw7"
server_key: "d05db343a7fa79d760f23373de006139"
server_key_iv: "6a289082ad8cd09322a3f728d32e476d"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:53.042377 27865 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.003s sys 0.000s
I20260807 06:39:53.069516 27884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:53.069773 27865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:39:53.069886 27865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "a010edb5de824625b20f44b4a783c467"
format_stamp: "Formatted at 2026-08-07 06:39:53 on dist-test-slave-gkw7"
server_key: "d05db343a7fa79d760f23373de006139"
server_key_iv: "6a289082ad8cd09322a3f728d32e476d"
server_key_version: "kuduclusterkey@0"
I20260807 06:39:53.070000 27865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:39:53.117156 27865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:39:53.120286 27865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:39:53.120471 27865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:39:53.120750 27865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:39:53.121212 27865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:39:53.121268 27865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:53.121306 27865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:39:53.121331 27865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:39:53.131829 27865 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:38613
I20260807 06:39:53.132364 27865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:39:53.133249 28037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:38613 every 8 connection(s)
I20260807 06:39:53.138310 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 27865
I20260807 06:39:53.138419 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:39:53.138612 11723 external_mini_cluster.cc:1428] Setting key fa7799698dd053fd4ad81959f42a4b13
Aug 07 06:39:53 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:39:53.145161 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.133724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:45297 (local address 127.11.114.254:36459)
0807 06:39:53.133807 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:53.133809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:53.134032 (+ 223us) server_negotiation.cc:407] Connection header received
0807 06:39:53.134788 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:53.134789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:53.134803 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:53.134838 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:53.135746 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.136520 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.138092 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.138407 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.138489 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.141513 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:53.141514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:53.141515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:53.141524 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:53.143398 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:53.143895 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:53.143897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:53.143897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:53.143964 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:53.144344 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:53.144345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:53.144345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:53.144490 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:53.144570 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:53.144965 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 06:39:53.145062 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:39:53.145330 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.133354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:36459 (local address 127.11.114.195:45297)
0807 06:39:53.133891 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:53.133941 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:53.134620 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:53.135041 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:53.135045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:53.135328 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:39:53.135612 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.135618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.136652 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.136653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.137983 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.137986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.138560 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.138562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.138707 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.139449 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:39:53.139459 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:39:53.141408 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:39:53.143491 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:53.143498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:53.143501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:53.143807 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:53.144042 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:39:53.144042 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:39:53.144043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:39:53.144261 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:39:53.144572 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:39:53.144578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:39:53.144746 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 06:39:53.144916 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 06:39:53.145081 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":91,"threads_started":1}
I20260807 06:39:53.146082 28040 heartbeater.cc:347] Connected to a master server at 127.11.114.254:36459
I20260807 06:39:53.146184 28040 heartbeater.cc:464] Registering TS with master...
I20260807 06:39:53.146375 28040 heartbeater.cc:511] Master 127.11.114.254:36459 requested a full tablet report, sending...
I20260807 06:39:53.146761 27381 ts_manager.cc:194] Registered new tserver with Master: a010edb5de824625b20f44b4a783c467 (127.11.114.195:38613)
I20260807 06:39:53.147274 27381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:45297
I20260807 06:39:53.150669 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:39:53.160583 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.152347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.11.114.254:36459)
0807 06:39:53.152386 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:53.152388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:53.152531 (+ 143us) server_negotiation.cc:407] Connection header received
0807 06:39:53.152655 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:53.152656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:53.152667 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:53.152698 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:39:53.153499 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.154268 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.155361 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.155655 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.155703 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.157458 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:39:53.157460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:39:53.157460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:39:53.157467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:39:53.159272 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:53.159697 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:53.159699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:53.159699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:53.159768 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:39:53.160001 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:39:53.160002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:39:53.160002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:39:53.160139 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:39:53.160245 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:53.160402 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 06:39:53.160493 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:39:53.162398 27381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:39:53.164736 27381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:39:53.179628 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.168177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38613 (local address 127.0.0.1:60518)
0807 06:39:53.168701 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:53.168709 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:53.168741 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:53.173827 (+ 5086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:53.173831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:53.173833 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:53.173907 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.173908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.175448 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.175449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.176866 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.176869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.179283 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.179284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.179368 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.179386 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 06:39:53.179447 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 06:39:53.179520 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":75,"threads_started":1}
I20260807 06:39:53.180402 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.168305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.11.114.195:38613)
0807 06:39:53.173477 (+ 5172us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:53.173483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:53.173496 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:39:53.173568 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:53.173570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:53.173590 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:53.173681 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:53.174022 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.175334 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.177011 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.179179 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.180124 (+ 945us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.180247 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:53.180277 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 06:39:53.180319 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5127,"thread_start_us":219,"threads_started":1}
I20260807 06:39:53.181296 27923 tablet_service.cc:1467] Processing CreateTablet for tablet 413e34b6127d4cb99ecac3d869d1792f (DEFAULT_TABLE table=test-table [id=45f630746866420d8e7550e43f569318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:39:53.181999 27923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 413e34b6127d4cb99ecac3d869d1792f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:53.186626 28065 tablet_bootstrap.cc:492] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: Bootstrap starting.
I20260807 06:39:53.187913 28065 tablet_bootstrap.cc:654] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:53.188325 28065 log.cc:824] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:53.189203 28065 tablet_bootstrap.cc:492] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: No bootstrap required, opened a new log
I20260807 06:39:53.189275 28065 ts_tablet_manager.cc:1397] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:39:53.191330 28065 raft_consensus.cc:348] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.191502 28065 raft_consensus.cc:374] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:53.191546 28065 raft_consensus.cc:729] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a010edb5de824625b20f44b4a783c467, State: Initialized, Role: FOLLOWER
I20260807 06:39:53.191676 28065 consensus_queue.cc:260] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.192004 28065 ts_tablet_manager.cc:1428] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:39:53.192358 28040 heartbeater.cc:503] Master 127.11.114.254:36459 was elected leader, sending a full tablet report...
I20260807 06:39:53.194150 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.167725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43927 (local address 127.0.0.1:47450)
0807 06:39:53.168347 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:53.168388 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:53.168456 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:53.189002 (+ 20546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:53.189004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:53.189007 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:53.189069 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.189071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.191037 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.191038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.192727 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.192731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.193893 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.193894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.193961 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.193965 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:39:53.194025 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:39:53.194068 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":82,"threads_started":1}
I20260807 06:39:53.194204 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.167828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47450 (local address 127.11.114.194:43927)
0807 06:39:53.175633 (+ 7805us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:53.175638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:53.175651 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:39:53.175693 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:53.175696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:53.175711 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:53.175790 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:53.189173 (+ 13383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.190314 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.192878 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.193708 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.193783 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.193845 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:53.194065 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 06:39:53.194112 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7758,"thread_start_us":79,"threads_started":1}
I20260807 06:39:53.195508 27764 tablet_service.cc:1467] Processing CreateTablet for tablet 413e34b6127d4cb99ecac3d869d1792f (DEFAULT_TABLE table=test-table [id=45f630746866420d8e7550e43f569318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:39:53.195780 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.167989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:39155 (local address 127.0.0.1:53954)
0807 06:39:53.168568 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:53.168578 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:53.168616 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:53.173835 (+ 5219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:53.173836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:53.173838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:53.173921 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.173923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.189807 (+ 15884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.189808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.190958 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.190961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.195514 (+ 4553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.195515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.195584 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.195588 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:39:53.195663 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 06:39:53.195708 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":72,"threads_started":1}
I20260807 06:39:53.195912 27764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 413e34b6127d4cb99ecac3d869d1792f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:53.196319 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.168089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53954 (local address 127.11.114.193:39155)
0807 06:39:53.173633 (+ 5544us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:53.173640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:53.173653 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:39:53.173692 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:53.173694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:53.173712 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:53.173796 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:53.174013 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.189698 (+ 15685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.194665 (+ 4967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.195409 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.195500 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.195568 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:53.196165 (+ 597us) server_negotiation.cc:299] Negotiation successful
0807 06:39:53.196219 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5501,"thread_start_us":65,"threads_started":1}
I20260807 06:39:53.198825 28074 tablet_bootstrap.cc:492] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: Bootstrap starting.
I20260807 06:39:53.200316 28074 tablet_bootstrap.cc:654] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:53.200291 27573 tablet_service.cc:1467] Processing CreateTablet for tablet 413e34b6127d4cb99ecac3d869d1792f (DEFAULT_TABLE table=test-table [id=45f630746866420d8e7550e43f569318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:39:53.200652 27573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 413e34b6127d4cb99ecac3d869d1792f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:39:53.200810 28074 log.cc:824] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: Log is configured to *not* fsync() on all Append() calls
I20260807 06:39:53.203248 28076 tablet_bootstrap.cc:492] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392: Bootstrap starting.
I20260807 06:39:53.204088 28076 tablet_bootstrap.cc:654] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392: Neither blocks nor log segments found. Creating new log.
I20260807 06:39:53.204836 28076 log.cc:824] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:39:53 dist-test-slave-gkw7 krb5kdc[26730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:39:53.205981 28076 tablet_bootstrap.cc:492] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392: No bootstrap required, opened a new log
I20260807 06:39:53.206056 28076 ts_tablet_manager.cc:1397] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:39:53.209091 28076 raft_consensus.cc:348] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.209261 28076 raft_consensus.cc:374] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:53.209297 28076 raft_consensus.cc:729] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e9455aee984504b830dfb178e90392, State: Initialized, Role: FOLLOWER
I20260807 06:39:53.209576 28076 consensus_queue.cc:260] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.209940 28076 ts_tablet_manager.cc:1428] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 06:39:53.210114 27663 heartbeater.cc:503] Master 127.11.114.254:36459 was elected leader, sending a full tablet report...
I20260807 06:39:53.210954 28074 tablet_bootstrap.cc:492] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: No bootstrap required, opened a new log
I20260807 06:39:53.211037 28074 ts_tablet_manager.cc:1397] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260807 06:39:53.213315 28074 raft_consensus.cc:348] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.213464 28074 raft_consensus.cc:374] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:39:53.213485 28074 raft_consensus.cc:729] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fce2a136a0ca4bd08bbfa09ba2b34033, State: Initialized, Role: FOLLOWER
I20260807 06:39:53.213693 28074 consensus_queue.cc:260] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.214136 27859 heartbeater.cc:503] Master 127.11.114.254:36459 was elected leader, sending a full tablet report...
I20260807 06:39:53.214115 28074 ts_tablet_manager.cc:1428] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:39:53.223918 28082 raft_consensus.cc:482] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:39:53.224053 28082 raft_consensus.cc:504] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.228271 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.224674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38613 (local address 127.11.114.194:56681)
0807 06:39:53.224785 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:53.224798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:53.224840 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:53.225129 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:53.225130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:53.225132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:53.225207 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.225209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.226158 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.226159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.227258 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.227261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.227992 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.227993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.228076 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.228080 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:39:53.228136 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:39:53.228179 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 06:39:53.228710 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.224927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:56681 (local address 127.11.114.195:38613)
0807 06:39:53.224974 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:53.224976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:53.224984 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:39:53.225014 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:53.225015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:53.225027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:53.225077 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:53.225295 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.226020 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.227363 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.227876 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.227946 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.227956 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:53.228561 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 06:39:53.228607 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:39:53.228991 27973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "413e34b6127d4cb99ecac3d869d1792f" candidate_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a010edb5de824625b20f44b4a783c467" is_pre_election: true
I20260807 06:39:53.229146 27973 raft_consensus.cc:2454] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fce2a136a0ca4bd08bbfa09ba2b34033 in term 0.
I20260807 06:39:53.229377 27697 leader_election.cc:302] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a010edb5de824625b20f44b4a783c467, fce2a136a0ca4bd08bbfa09ba2b34033; no voters:
I20260807 06:39:53.231824 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.225010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:39155 (local address 127.11.114.194:44391)
0807 06:39:53.228346 (+ 3336us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:53.228357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:39:53.228413 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:39:53.228591 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:39:53.228592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:39:53.228593 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:39:53.228621 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.228622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.229654 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.229655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.230579 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:39:53.230582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.231512 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.231513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:39:53.231680 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.231684 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:39:53.231730 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 06:39:53.231762 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3326,"thread_start_us":128,"threads_started":1}
I20260807 06:39:53.232206 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.225267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:44391 (local address 127.11.114.193:39155)
0807 06:39:53.225313 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:53.225315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:53.228404 (+ 3089us) server_negotiation.cc:407] Connection header received
0807 06:39:53.228477 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:53.228479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:53.228494 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:53.228545 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:39:53.228689 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.229336 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.230683 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.231218 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.231301 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.231308 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:53.232079 (+ 771us) server_negotiation.cc:299] Negotiation successful
0807 06:39:53.232128 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:39:53.232501 27596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "413e34b6127d4cb99ecac3d869d1792f" candidate_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e9455aee984504b830dfb178e90392" is_pre_election: true
I20260807 06:39:53.232676 27596 raft_consensus.cc:2454] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fce2a136a0ca4bd08bbfa09ba2b34033 in term 0.
I20260807 06:39:53.234025 28082 leader_election.cc:288] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34e9455aee984504b830dfb178e90392 (127.11.114.193:39155), a010edb5de824625b20f44b4a783c467 (127.11.114.195:38613)
I20260807 06:39:53.234155 28082 raft_consensus.cc:2781] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:39:53.234217 28082 raft_consensus.cc:482] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:39:53.234252 28082 raft_consensus.cc:3037] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:53.235203 28082 raft_consensus.cc:504] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.235517 27596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "413e34b6127d4cb99ecac3d869d1792f" candidate_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e9455aee984504b830dfb178e90392"
I20260807 06:39:53.235526 27973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "413e34b6127d4cb99ecac3d869d1792f" candidate_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a010edb5de824625b20f44b4a783c467"
I20260807 06:39:53.235610 27596 raft_consensus.cc:3037] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:53.235622 27973 raft_consensus.cc:3037] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:39:53.236583 27973 raft_consensus.cc:2454] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fce2a136a0ca4bd08bbfa09ba2b34033 in term 1.
I20260807 06:39:53.236687 27596 raft_consensus.cc:2454] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fce2a136a0ca4bd08bbfa09ba2b34033 in term 1.
I20260807 06:39:53.236701 28082 leader_election.cc:288] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [CANDIDATE]: Term 1 election: Requested vote from peers 34e9455aee984504b830dfb178e90392 (127.11.114.193:39155), a010edb5de824625b20f44b4a783c467 (127.11.114.195:38613)
I20260807 06:39:53.236846 27697 leader_election.cc:302] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a010edb5de824625b20f44b4a783c467, fce2a136a0ca4bd08bbfa09ba2b34033; no voters:
I20260807 06:39:53.236960 28082 raft_consensus.cc:2781] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:39:53.237031 28082 raft_consensus.cc:686] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 1 LEADER]: Becoming Leader. State: Replica: fce2a136a0ca4bd08bbfa09ba2b34033, State: Running, Role: LEADER
I20260807 06:39:53.237155 28082 consensus_queue.cc:237] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } }
I20260807 06:39:53.238978 27380 catalog_manager.cc:5495] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 reported cstate change: term changed from 0 to 1, leader changed from <none> to fce2a136a0ca4bd08bbfa09ba2b34033 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce2a136a0ca4bd08bbfa09ba2b34033" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260807 06:39:53.249410 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.245927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.11.114.194:43927)
0807 06:39:53.246001 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:39:53.246004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:39:53.246019 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:39:53.246082 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:39:53.246083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:39:53.246104 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:39:53.246184 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:39:53.246358 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.247073 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.248164 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:39:53.248496 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:39:53.248562 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:39:53.248817 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:39:53.248948 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:39:53.249036 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:39:53.249290 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 06:39:53.249313 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:39:53.251925 27973 raft_consensus.cc:1261] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 1 FOLLOWER]: Refusing update from remote peer fce2a136a0ca4bd08bbfa09ba2b34033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:39:53.252365 28086 consensus_queue.cc:1035] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a010edb5de824625b20f44b4a783c467" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:39:53.254943 27596 raft_consensus.cc:1261] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392 [term 1 FOLLOWER]: Refusing update from remote peer fce2a136a0ca4bd08bbfa09ba2b34033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:39:53.255611 28086 consensus_queue.cc:1035] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e9455aee984504b830dfb178e90392" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:39:53.257085 28089 mvcc.cc:205] Tried to move back new op lower bound from 7315803313156571136 to 7315803313103421440. Current Snapshot: MvccSnapshot[applied={T|T < 7315803313156571136}]
W20260807 06:39:53.257313 27664 tablet.cc:2369] T 413e34b6127d4cb99ecac3d869d1792f P 34e9455aee984504b830dfb178e90392: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:39:53.257936 28090 mvcc.cc:205] Tried to move back new op lower bound from 7315803313156571136 to 7315803313103421440. Current Snapshot: MvccSnapshot[applied={T|T < 7315803313156571136}]
I20260807 06:39:53.261340 27380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:39:53.261453 27380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:39:53.262367 27380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e02d9e21667b4f9ebd1f6abf70377540: Sending DeleteTablet for 3 replicas of tablet 413e34b6127d4cb99ecac3d869d1792f
I20260807 06:39:53.262914 27573 tablet_service.cc:1514] Processing DeleteTablet for tablet 413e34b6127d4cb99ecac3d869d1792f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:39:53 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:53954
I20260807 06:39:53.263056 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 27488
I20260807 06:39:53.263196 27923 tablet_service.cc:1514] Processing DeleteTablet for tablet 413e34b6127d4cb99ecac3d869d1792f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:39:53 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:60518
I20260807 06:39:53.263710 27764 tablet_service.cc:1514] Processing DeleteTablet for tablet 413e34b6127d4cb99ecac3d869d1792f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:39:53 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:47450
I20260807 06:39:53.280707 28104 tablet_replica.cc:330] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: stopping tablet replica
I20260807 06:39:53.280905 28104 raft_consensus.cc:2229] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:39:53.281051 28104 raft_consensus.cc:2258] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:39:53.281265 28102 tablet_replica.cc:330] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: stopping tablet replica
I20260807 06:39:53.281385 28102 raft_consensus.cc:2229] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:39:53.281517 28102 raft_consensus.cc:2258] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:39:53.282327 28104 ts_tablet_manager.cc:1905] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:39:53.284390 28102 ts_tablet_manager.cc:1905] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 06:39:53.289106 27701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:39155 (error 108)
I20260807 06:39:53.290190 28102 ts_tablet_manager.cc:1918] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:39:53.290282 28102 log.cc:1196] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/413e34b6127d4cb99ecac3d869d1792f
I20260807 06:39:53.290912 28102 ts_tablet_manager.cc:1939] T 413e34b6127d4cb99ecac3d869d1792f P a010edb5de824625b20f44b4a783c467: Deleting consensus metadata
I20260807 06:39:53.291406 27360 catalog_manager.cc:4841] TS a010edb5de824625b20f44b4a783c467 (127.11.114.195:38613): tablet 413e34b6127d4cb99ecac3d869d1792f (table test-table [id=45f630746866420d8e7550e43f569318]) successfully deleted
I20260807 06:39:53.292389 28104 ts_tablet_manager.cc:1918] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:39:53.292552 28104 log.cc:1196] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/413e34b6127d4cb99ecac3d869d1792f
I20260807 06:39:53.293210 28104 ts_tablet_manager.cc:1939] T 413e34b6127d4cb99ecac3d869d1792f P fce2a136a0ca4bd08bbfa09ba2b34033: Deleting consensus metadata
I20260807 06:39:53.293764 27364 catalog_manager.cc:4841] TS fce2a136a0ca4bd08bbfa09ba2b34033 (127.11.114.194:43927): tablet 413e34b6127d4cb99ecac3d869d1792f (table test-table [id=45f630746866420d8e7550e43f569318]) successfully deleted
W20260807 06:39:53.294090 27365 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:39155 (error 108)
I20260807 06:39:53.294935 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 27675
I20260807 06:39:53.330930 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:39:53.330749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:39155 (local address 127.0.0.1:53964)
0807 06:39:53.330822 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 06:39:53.330881 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 06:39:53.331168 27365 catalog_manager.cc:4584] TS 34e9455aee984504b830dfb178e90392 (127.11.114.193:39155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 413e34b6127d4cb99ecac3d869d1792f: Network error: Client connection negotiation failed: client connection to 127.11.114.193:39155: connect: Connection refused (error 111)
I20260807 06:39:53.337229 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 27865
I20260807 06:39:53.347680 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 27340
I20260807 06:39:53.356065 11723 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 06:39:53.701259 11723 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 06:39:53.701398 11723 mini_ranger.cc:66] Stopping Ranger...
I20260807 06:39:54.056636 11723 mini_ranger.cc:68] Stopped Ranger
2026-08-07 06:39:54.057 UTC [26770] LOG: received smart shutdown request
2026-08-07 06:39:54.060 UTC [26770] LOG: background worker "logical replication launcher" (PID 26783) exited with exit code 1
2026-08-07 06:39:54.061 UTC [26775] LOG: shutting down
2026-08-07 06:39:54.219 UTC [26770] LOG: database system is shut down
2026-08-07T06:39:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47726 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:39:54 dist-test-slave-gkw7 krb5kdc[28123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:39:54 dist-test-slave-gkw7 krb5kdc[28123](info): set up 2 sockets
Aug 07 06:39:54 dist-test-slave-gkw7 krb5kdc[28123](info): commencing operation
krb5kdc: starting...
W20260807 06:39:55.621016 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:39:55 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:39:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:39:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.114.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.114.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/rangeradmin_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/rangeradmin_127.11.114.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.114.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.114.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/rangerlookup_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/rangerlookup_127.11.114.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.114.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.114.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/HTTP_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/HTTP_127.11.114.212@KRBTEST.COM.keytab.
I20260807 06:39:55.794436 11723 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZbZY4B/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/postgres -l logfile start
2026-08-07 06:39:58.358 UTC [28165] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 06:39:58.358 UTC [28165] LOG: listening on IPv4 address "127.11.114.212", port 59079
2026-08-07 06:39:58.362 UTC [28165] LOG: listening on Unix socket "/tmp/.s.PGSQL.59079"
2026-08-07 06:39:58.397 UTC [28169] LOG: database system was shut down at 2026-08-07 06:39:56 UTC
2026-08-07 06:39:58.401 UTC [28165] LOG: database system is ready to accept connections
I20260807 06:39:59.517721 11723 mini_postgres.cc:96] Postgres bound to 59079
2026-08-07 06:39:59.525 UTC [28180] FATAL: database "slave" does not exist
127.11.114.212:59079 - accepting connections
I20260807 06:39:59.525763 11723 mini_ranger.cc:161] Starting Ranger...
I20260807 06:39:59.541791 11723 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 06:40:00.049639 11723 mini_ranger.cc:87] Created ranger Postgres database
I20260807 06:40:00.049714 11723 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-admin
2026-08-07 06:40:00,345 [I] DB FLAVOR :POSTGRES
2026-08-07 06:40:00,346 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:40:00,346 [I] Checking connection..
2026-08-07 06:40:00,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 06:40:00,632 [I] Checking connection passed.
2026-08-07 06:40:00,633 [I] --------- Verifying version history table ---------
2026-08-07 06:40:00,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:40:00,907 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:40:00,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:40:01,167 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:40:01,167 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 06:40:01,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 06:40:01.444 UTC [28259] WARNING: there is no transaction in progress
2026-08-07 06:40:01,452 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 06:40:01,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:40:01,724 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 06:40:01,725 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:40:01,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 06:40:01,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 06:40:02,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-07 06:40:02,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:40:02,757 [I] Table x_portal_user does not exist in database ranger
2026-08-07 06:40:02,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:40:03,012 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 06:40:03,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:40:03,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:40:03,532 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 06:40:03,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 06:40:04.561 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.579 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.596 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.612 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.631 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.649 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.704 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.710 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.725 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.735 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.746 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.761 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.771 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.779 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.787 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:04.794 UTC [28411] WARNING: there is no transaction in progress
2026-08-07 06:40:05,476 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 06:40:05,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 06:40:05,733 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 06:40:05,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:40:05,975 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 06:40:05,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:40:06,250 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 06:40:06,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:40:06,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:40:06,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-07 06:40:07,006 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 06:40:07,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 06:40:07,248 [I] DB_PATCHES have already been applied
I20260807 06:40:07.254662 11723 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZbZY4B/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 06:40:07.254758 11723 mini_ranger.cc:193] Using host: 127.11.114.212
I20260807 06:40:07.258630 11723 mini_ranger.cc:227] Ranger admin URL: http://127.11.114.212:58085
Aug 07, 2026 6:40:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:40:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:40:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:40:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:40:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:40:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:40:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 06:40:08.649940 11723 mini_ranger.cc:160] Time spent starting Ranger: real 9.124s user 0.000s sys 0.010s
Aug 07, 2026 6:40:30 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:40:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 6:40:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:40:30 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:40:31 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 06:40:31.807986 11723 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.114.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.114.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/rangerkms_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.114.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/rangerkms_127.11.114.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.114.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.114.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.114.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/HTTP_127.11.114.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.114.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/HTTP_127.11.114.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 06:40:31 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084831, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 06:40:31.898334 11723 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 06:40:31.910004 11723 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 06:40:32.832620 11723 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 06:40:32.832723 11723 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms
I20260807 06:40:32.832747 11723 mini_ranger_kms.cc:227] Using postgres at 127.11.114.212:59079
2026-08-07 06:40:33,021 [I] DB FLAVOR :POSTGRES
2026-08-07 06:40:33,021 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:40:33,021 [I] Checking connection
2026-08-07 06:40:33,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 06:40:33,304 [I] connection success
2026-08-07 06:40:33,305 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 06:40:33,305 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 06:40:33,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 06:40:33,572 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 06:40:33,572 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:40:33,572 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 06:40:33,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.114.212:59079/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 06:40:33,849 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 06:40:34.345937 11723 mini_ranger_kms.cc:310] Created kms service
I20260807 06:40:34.574733 11723 mini_ranger_kms.cc:326] Created kudu user
I20260807 06:40:34.624230 11723 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 06:40:34 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084831, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:35.052132 11723 mini_ranger_kms.cc:384] Added ranger policy
I20260807 06:40:35.052371 11723 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZbZY4B/build/release/bin/postgresql.jar:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZbZY4B/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 06:40:35.052395 11723 mini_ranger_kms.cc:240] Using host: 127.11.114.212
I20260807 06:40:35.056958 11723 mini_ranger_kms.cc:276] Ranger KMS PID: 28646
I20260807 06:40:35.057061 11723 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.114.212:52271
Aug 07, 2026 6:40:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:40:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:40:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:40:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:40:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:40:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 06:40:37 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.114.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:37.867214 11723 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.969s user 0.000s sys 0.171s
I20260807 06:40:37.867324 11723 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 06:40:37.867375 11723 mini_ranger_kms.cc:401] 127.11.114.212:52271/kms/v1/keys
Aug 07 06:40:37 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.114.212@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
Aug 07 06:40:38 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.114.212@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
Aug 07 06:40:40 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:40:40.637477 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39213
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39213
--ranger_config_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:52271/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:40.757923 28712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:40.758136 28712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:40.758158 28712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:40.759352 28712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:40.759398 28712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:40.759413 28712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:40.759425 28712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:40.759438 28712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:40.761034 28712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.114.212:52271/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39213
--ranger_config_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:40.761284 28712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:40.761581 28712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:40.768543 28721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:40.768672 28723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:40.768738 28712 server_base.cc:1034] running on GCE node
W20260807 06:40:40.768570 28719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:40.769050 28712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:40.769380 28712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:40.770540 28712 hybrid_clock.cc:648] HybridClock initialized: now 1786084840770516 us; error 46 us; skew 500 ppm
Aug 07 06:40:40 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:40.777212 28712 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:40:40.777786 28712 webserver.cc:460] Webserver started at http://127.11.114.254:39261/ using document root <none> and password file <none>
I20260807 06:40:40.778019 28712 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:40.778120 28712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:40.778287 28712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:40:40 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:40.846207 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "cd6aeb6192bb48b49eb69a9d7653d105"
format_stamp: "Formatted at 2026-08-07 06:40:40 on dist-test-slave-gkw7"
server_key: "94b0fbe476cc11574a1f60ed129ab55a"
server_key_iv: "411922588a91d512782fec40cec6dc89"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:40.846832 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "cd6aeb6192bb48b49eb69a9d7653d105"
format_stamp: "Formatted at 2026-08-07 06:40:40 on dist-test-slave-gkw7"
server_key: "94b0fbe476cc11574a1f60ed129ab55a"
server_key_iv: "411922588a91d512782fec40cec6dc89"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:40.849087 28712 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 06:40:40.865688 28730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:40.865988 28712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:40:40.866123 28712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "cd6aeb6192bb48b49eb69a9d7653d105"
format_stamp: "Formatted at 2026-08-07 06:40:40 on dist-test-slave-gkw7"
server_key: "94b0fbe476cc11574a1f60ed129ab55a"
server_key_iv: "411922588a91d512782fec40cec6dc89"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:40.866261 28712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:40.886648 28712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:40.888845 28712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:40.889034 28712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:40.894232 28712 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39213
I20260807 06:40:40.894253 28809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39213 every 8 connection(s)
I20260807 06:40:40.894708 28712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:40:40.895608 28810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:40.895700 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 28712
I20260807 06:40:40.895808 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:40:40.895982 11723 external_mini_cluster.cc:1428] Setting key be9ad1ce5ce63b7d60354ac738b09f70
I20260807 06:40:40.899475 28810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Bootstrap starting.
I20260807 06:40:40.900285 28810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:40.900620 28810 log.cc:824] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:40.901924 28810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: No bootstrap required, opened a new log
Aug 07 06:40:40 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:40.904474 28810 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } }
I20260807 06:40:40.904652 28810 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:40.904718 28810 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6aeb6192bb48b49eb69a9d7653d105, State: Initialized, Role: FOLLOWER
I20260807 06:40:40.904901 28810 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } }
I20260807 06:40:40.905022 28810 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:40:40.905073 28810 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:40:40.905174 28810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:40.905803 28810 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } }
I20260807 06:40:40.905927 28810 leader_election.cc:302] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd6aeb6192bb48b49eb69a9d7653d105; no voters:
I20260807 06:40:40.906136 28810 leader_election.cc:288] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:40:40.906167 28818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:40:40.906371 28818 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 1 LEADER]: Becoming Leader. State: Replica: cd6aeb6192bb48b49eb69a9d7653d105, State: Running, Role: LEADER
I20260807 06:40:40.906451 28810 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:40.906499 28818 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } }
I20260807 06:40:40.906900 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } } }
I20260807 06:40:40.907155 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:40.907337 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd6aeb6192bb48b49eb69a9d7653d105. Latest consensus state: current_term: 1 leader_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } } }
I20260807 06:40:40.907459 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:40.907680 28830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:40:40.907975 28810 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 06:40:40.908293 28830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:40:40.911669 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:40.896523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.11.114.254:39213)
0807 06:40:40.896686 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:40.896696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:40.896787 (+ 91us) server_negotiation.cc:407] Connection header received
0807 06:40:40.897320 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:40.897327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:40.897377 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:40.897536 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:40.898277 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:40.899189 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:40.900044 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:40.900517 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:40.900636 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:40.903885 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:40.903890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:40.903890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:40.903904 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:40.906058 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:40.906570 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:40.906573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:40.906574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:40.906694 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:40.906924 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:40.906925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:40.906925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:40.907305 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:40.907957 (+ 652us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:40.908057 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 06:40:40.908200 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260807 06:40:40.922073 28830 catalog_manager.cc:1347] Generated new cluster ID: f35d7393ed984adebd7fe15d99caab59
I20260807 06:40:40.922144 28830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:40:40.945631 28830 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:40:40.946583 28830 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:40:40.961632 28830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Generated new TSK 0
I20260807 06:40:40.962553 28830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:40:41.679764 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:52271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39213
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:41.822938 28874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:41.823273 28874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:41.823321 28874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:41.825745 28874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:41.825810 28874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:41.825858 28874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:40:41.828317 28874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.114.212:52271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:41.828606 28874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:41.828881 28874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:41.836103 28880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:41.836426 28884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:41.842289 28879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:41.842507 28874 server_base.cc:1034] running on GCE node
I20260807 06:40:41.842790 28874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:41.843190 28874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:41.846740 28874 hybrid_clock.cc:648] HybridClock initialized: now 1786084841846717 us; error 41 us; skew 500 ppm
Aug 07 06:40:41 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084841, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:41.851962 28874 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:40:41.852569 28874 webserver.cc:460] Webserver started at http://127.11.114.193:46003/ using document root <none> and password file <none>
I20260807 06:40:41.852842 28874 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:41.852900 28874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:41.853034 28874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:40:41 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084841, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:41.871227 28874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "88b2142f44d94ed8bf006036815eaab6"
format_stamp: "Formatted at 2026-08-07 06:40:41 on dist-test-slave-gkw7"
server_key: "5bf12f7bda8e264747de9649912495c5"
server_key_iv: "d8aea84983c2dfd65a71c6479d01e1d9"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:41.871718 28874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "88b2142f44d94ed8bf006036815eaab6"
format_stamp: "Formatted at 2026-08-07 06:40:41 on dist-test-slave-gkw7"
server_key: "5bf12f7bda8e264747de9649912495c5"
server_key_iv: "d8aea84983c2dfd65a71c6479d01e1d9"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:41.874037 28874 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 06:40:41.888474 28899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:41.888660 28874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:40:41.888764 28874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "88b2142f44d94ed8bf006036815eaab6"
format_stamp: "Formatted at 2026-08-07 06:40:41 on dist-test-slave-gkw7"
server_key: "5bf12f7bda8e264747de9649912495c5"
server_key_iv: "d8aea84983c2dfd65a71c6479d01e1d9"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:41.888875 28874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:41.920236 28874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:41.922864 28874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:41.923032 28874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:41.923318 28874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:41.923732 28874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:41.923795 28874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:41.923837 28874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:41.923861 28874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:41.934587 28874 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:36323
I20260807 06:40:41.935101 28874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:40:41.937918 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 28874
I20260807 06:40:41.938035 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:40:41.938248 11723 external_mini_cluster.cc:1428] Setting key 71db0551f0a40c6d6df4bc63bb0ebfef
I20260807 06:40:41.938660 29055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:36323 every 8 connection(s)
Aug 07 06:40:41 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084841, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:41.948127 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:41.936004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39213 (local address 127.11.114.193:40831)
0807 06:40:41.936227 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:41.936282 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:41.936962 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:41.937473 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:41.937478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:41.937813 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:41.938117 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:41.938124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:41.939629 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:41.939630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:41.940991 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:41.940994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:41.941617 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:41.941618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:41.941791 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:41.942488 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:41.942500 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:41.944103 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:41.946176 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:41.946184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:41.946187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:41.946439 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:41.946708 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:41.946709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:41.946709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:41.946866 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:41.947259 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:41.947267 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:41.947460 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 06:40:41.947593 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 06:40:41.947787 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":98,"threads_started":1}
I20260807 06:40:41.948670 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:41.936323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:40831 (local address 127.11.114.254:39213)
0807 06:40:41.937181 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:41.937183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:41.937193 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:40:41.937223 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:41.937224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:41.937238 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:41.937339 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:41.938241 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:41.939502 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:41.941098 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:41.941424 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:41.941496 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:41.944221 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:41.944223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:41.944223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:41.944231 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:41.946060 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:41.946543 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:41.946544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:41.946544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:41.946623 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:41.946959 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:41.946960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:41.946960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:41.947146 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:41.947223 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:41.948449 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0807 06:40:41.948549 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":75,"threads_started":1}
I20260807 06:40:41.949301 29056 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39213
I20260807 06:40:41.949415 29056 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:41.949677 29056 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
I20260807 06:40:41.950335 28758 ts_manager.cc:194] Registered new tserver with Master: 88b2142f44d94ed8bf006036815eaab6 (127.11.114.193:36323)
I20260807 06:40:41.951330 28758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:40831
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:40:41.979192 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:52271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39213
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 06:40:42.110862 29064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:42.111245 29064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:42.111334 29064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:42.113114 29064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:42.113235 29064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:42.113323 29064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:40:42.115664 29064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.114.212:52271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:42.116142 29064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:42.116580 29064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:42.126665 29077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:42.126863 29080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:42.126663 29074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:42.127487 29064 server_base.cc:1034] running on GCE node
I20260807 06:40:42.127722 29064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:42.127985 29064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:42.129151 29064 hybrid_clock.cc:648] HybridClock initialized: now 1786084842129127 us; error 38 us; skew 500 ppm
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:42.131835 29064 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:40:42.132283 29064 webserver.cc:460] Webserver started at http://127.11.114.194:45423/ using document root <none> and password file <none>
I20260807 06:40:42.132551 29064 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:42.132623 29064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:42.132784 29064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:42.146430 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "3d882df0fc0747d0986c6aab9aa7ee29"
format_stamp: "Formatted at 2026-08-07 06:40:42 on dist-test-slave-gkw7"
server_key: "00533cb6c648efca287a8ecf23d8dd34"
server_key_iv: "3888ee7d99104b1f256dd7f4954d8196"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:42.146927 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "3d882df0fc0747d0986c6aab9aa7ee29"
format_stamp: "Formatted at 2026-08-07 06:40:42 on dist-test-slave-gkw7"
server_key: "00533cb6c648efca287a8ecf23d8dd34"
server_key_iv: "3888ee7d99104b1f256dd7f4954d8196"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:42.148542 29064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:40:42.160758 29089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.160989 29064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:42.161089 29064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "3d882df0fc0747d0986c6aab9aa7ee29"
format_stamp: "Formatted at 2026-08-07 06:40:42 on dist-test-slave-gkw7"
server_key: "00533cb6c648efca287a8ecf23d8dd34"
server_key_iv: "3888ee7d99104b1f256dd7f4954d8196"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:42.161201 29064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:42.170686 29064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:42.173053 29064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:42.173273 29064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:42.173612 29064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:42.174031 29064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:42.174129 29064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.174206 29064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:42.174284 29064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.181890 29064 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:36679
I20260807 06:40:42.182363 29064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:40:42.182936 29262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:36679 every 8 connection(s)
I20260807 06:40:42.184671 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 29064
I20260807 06:40:42.184787 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:40:42.185022 11723 external_mini_cluster.cc:1428] Setting key 2a79169cec62c5e00250a4e509f2f71e
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:42.194593 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:42.183503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:48951 (local address 127.11.114.254:39213)
0807 06:40:42.183553 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:42.183555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:42.183568 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:40:42.184141 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:42.184142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:42.184159 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:42.184227 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:42.184970 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.185836 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.187366 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.187684 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.187725 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:42.190699 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:42.190701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:42.190701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:42.190709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:42.192556 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:42.193207 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:42.193209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:42.193210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:42.193292 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:42.193721 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:42.193722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:42.193723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:42.193884 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:42.193932 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:42.194355 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 06:40:42.194456 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:40:42.195178 29265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:42.183164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39213 (local address 127.11.114.194:48951)
0807 06:40:42.183354 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:42.183402 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:42.184023 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:42.184308 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:42.184312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:42.184571 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:42.184848 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:42.184854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.185954 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.185955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:42.187252 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:42.187265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.187830 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.187831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:42.187979 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:42.188812 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:42.188822 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:42.190589 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:42.192680 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:42.192688 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:42.192691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:42.193100 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:42.193380 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:42.193381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:42.193381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:42.193625 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:42.193984 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:42.193991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:42.194173 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 06:40:42.194737 (+ 564us) client_negotiation.cc:240] Negotiation successful
0807 06:40:42.194922 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260807 06:40:42.195907 29263 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39213
I20260807 06:40:42.196024 29263 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:42.196239 29263 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
I20260807 06:40:42.196681 28758 ts_manager.cc:194] Registered new tserver with Master: 3d882df0fc0747d0986c6aab9aa7ee29 (127.11.114.194:36679)
I20260807 06:40:42.197146 28758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:48951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:40:42.214565 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:52271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39213
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:42.309795 29268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:42.310020 29268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:42.310058 29268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:42.311201 29268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:42.311268 29268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:42.311312 29268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:40:42.312803 29268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.114.212:52271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:42.313056 29268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:42.313323 29268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:42.318691 29275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:42.318717 29273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:42.319018 29279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:42.319283 29268 server_base.cc:1034] running on GCE node
I20260807 06:40:42.319483 29268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:42.319767 29268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:42.320936 29268 hybrid_clock.cc:648] HybridClock initialized: now 1786084842320924 us; error 27 us; skew 500 ppm
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:42.324004 29268 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:40:42.324522 29268 webserver.cc:460] Webserver started at http://127.11.114.195:40119/ using document root <none> and password file <none>
I20260807 06:40:42.324805 29268 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:42.324860 29268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:42.325038 29268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:42.343755 29268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "a0b928d793f24b69a78dc3d2e095c8ac"
format_stamp: "Formatted at 2026-08-07 06:40:42 on dist-test-slave-gkw7"
server_key: "d5786eeefb7da48bdcd6178bd4a2dcf0"
server_key_iv: "f3f50f8d51e32091674ecbeca21a9bbd"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:42.344302 29268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "a0b928d793f24b69a78dc3d2e095c8ac"
format_stamp: "Formatted at 2026-08-07 06:40:42 on dist-test-slave-gkw7"
server_key: "d5786eeefb7da48bdcd6178bd4a2dcf0"
server_key_iv: "f3f50f8d51e32091674ecbeca21a9bbd"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:42.347626 29268 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 06:40:42.363373 29293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.363698 29268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 06:40:42.363804 29268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "a0b928d793f24b69a78dc3d2e095c8ac"
format_stamp: "Formatted at 2026-08-07 06:40:42 on dist-test-slave-gkw7"
server_key: "d5786eeefb7da48bdcd6178bd4a2dcf0"
server_key_iv: "f3f50f8d51e32091674ecbeca21a9bbd"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:42.363900 29268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:42.377707 29268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:42.379850 29268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:42.380020 29268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:42.380283 29268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:42.380640 29268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:42.380698 29268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.380766 29268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:42.380816 29268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.386803 29268 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:39633
I20260807 06:40:42.387293 29268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:40:42.389993 29459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:39633 every 8 connection(s)
I20260807 06:40:42.390913 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 29268
I20260807 06:40:42.391032 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:40:42.391551 11723 external_mini_cluster.cc:1428] Setting key ff5244c4d1578ea1f6fc3da1fe88f6da
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:42.400645 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:42.390655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:42545 (local address 127.11.114.254:39213)
0807 06:40:42.390725 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:42.390727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:42.391268 (+ 541us) server_negotiation.cc:407] Connection header received
0807 06:40:42.391998 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:42.391999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:42.392012 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:42.392057 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:42.392842 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.393468 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.394525 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.394734 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.394824 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:42.397268 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:42.397269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:42.397269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:42.397277 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:42.398896 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:42.399431 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:42.399432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:42.399433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:42.399538 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:42.399904 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:42.399905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:42.399906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:42.400034 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:42.400178 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:42.400468 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 06:40:42.400553 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:40:42.400847 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:42.390625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39213 (local address 127.11.114.195:42545)
0807 06:40:42.391119 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:42.391171 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:42.391897 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:42.392130 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:42.392134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:42.392436 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:42.392737 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:42.392744 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.393582 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.393584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:42.394413 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:42.394415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.394852 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.394853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:42.394988 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:42.395601 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:42.395612 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:42.397160 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:42.398969 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:42.398977 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:42.398980 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:42.399344 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:42.399627 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:42.399628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:42.399629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:42.399831 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:42.400171 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:42.400178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:42.400328 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 06:40:42.400435 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 06:40:42.400601 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":67,"threads_started":1}
I20260807 06:40:42.401326 29461 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39213
I20260807 06:40:42.401423 29461 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:42.401652 29461 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
I20260807 06:40:42.402134 28758 ts_manager.cc:194] Registered new tserver with Master: a0b928d793f24b69a78dc3d2e095c8ac (127.11.114.195:39633)
I20260807 06:40:42.402515 28758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:42545
I20260807 06:40:42.409956 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:40:42.410681 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 28874
I20260807 06:40:42.428285 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 29064
I20260807 06:40:42.454735 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 29268
I20260807 06:40:42.474915 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 28712
I20260807 06:40:42.484746 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39213
--webserver_interface=127.11.114.254
--webserver_port=39261
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39213
--ranger_config_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:52271/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.114.212:52271/kms with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:42.569839 29464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:42.570055 29464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:42.570078 29464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:42.571311 29464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:42.571360 29464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:42.571374 29464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:42.571388 29464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:42.571399 29464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:42.573035 29464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.114.212:52271/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39213
--ranger_config_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:42.573311 29464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:42.573582 29464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:42.578838 29473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:42.578930 29464 server_base.cc:1034] running on GCE node
W20260807 06:40:42.578809 29474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:42.579068 29477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:42.579339 29464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:42.579672 29464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:42.580838 29464 hybrid_clock.cc:648] HybridClock initialized: now 1786084842580818 us; error 41 us; skew 500 ppm
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:42.583477 29464 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:40:42.583940 29464 webserver.cc:460] Webserver started at http://127.11.114.254:39261/ using document root <none> and password file <none>
I20260807 06:40:42.584129 29464 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:42.584175 29464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:42.631032 29464 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:42.632128 29489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.632458 29464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:42.632572 29464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "cd6aeb6192bb48b49eb69a9d7653d105"
format_stamp: "Formatted at 2026-08-07 06:40:40 on dist-test-slave-gkw7"
server_key: "94b0fbe476cc11574a1f60ed129ab55a"
server_key_iv: "411922588a91d512782fec40cec6dc89"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:42.633040 29464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:42.648849 29464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:42.651221 29464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:42.651379 29464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:42.656726 29464 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39213
I20260807 06:40:42.656908 29564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39213 every 8 connection(s)
I20260807 06:40:42.657315 29464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:40:42.658690 29566 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:40:42.659379 29566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Bootstrap starting.
I20260807 06:40:42.661881 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 29464
I20260807 06:40:42.662223 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:52271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:36323
--local_ip_for_outbound_sockets=127.11.114.193
--tserver_master_addrs=127.11.114.254:39213
--webserver_port=46003
--webserver_interface=127.11.114.193
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.114.212:52271/kms with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:40:42.669152 29566 log.cc:824] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:42.671865 29566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:40:42.672235 29566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Bootstrap complete.
I20260807 06:40:42.676350 29566 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } }
I20260807 06:40:42.676712 29566 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6aeb6192bb48b49eb69a9d7653d105, State: Initialized, Role: FOLLOWER
I20260807 06:40:42.676890 29566 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } }
I20260807 06:40:42.676956 29566 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:40:42.676993 29566 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:40:42.677039 29566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:40:42.678421 29566 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } }
I20260807 06:40:42.678539 29566 leader_election.cc:302] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd6aeb6192bb48b49eb69a9d7653d105; no voters:
I20260807 06:40:42.678733 29566 leader_election.cc:288] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:40:42.678830 29571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:40:42.678990 29566 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:42.678996 29571 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [term 2 LEADER]: Becoming Leader. State: Replica: cd6aeb6192bb48b49eb69a9d7653d105, State: Running, Role: LEADER
I20260807 06:40:42.679209 29571 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } }
I20260807 06:40:42.679581 29575 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd6aeb6192bb48b49eb69a9d7653d105. Latest consensus state: current_term: 2 leader_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } } }
I20260807 06:40:42.679694 29575 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:42.679581 29573 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6aeb6192bb48b49eb69a9d7653d105" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39213 } } }
I20260807 06:40:42.679920 29573 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:42.680039 29587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:40:42.680653 29566 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 06:40:42.680794 29587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:40:42.680939 29587 catalog_manager.cc:1259] Loaded cluster ID: f35d7393ed984adebd7fe15d99caab59
I20260807 06:40:42.680981 29587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:40:42.685472 29587 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:40:42.685873 29587 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Loaded TSK: 0
I20260807 06:40:42.686113 29587 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:40:42.785786 29568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:42.786077 29568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:42.786126 29568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:42.787820 29568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:42.787873 29568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:42.787912 29568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:40:42.790082 29568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.114.212:52271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:36323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=46003
--tserver_master_addrs=127.11.114.254:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:42.790314 29568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:42.790550 29568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:42.797326 29628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:42.797619 29626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:42.797302 29631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:42.798951 29568 server_base.cc:1034] running on GCE node
I20260807 06:40:42.799211 29568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:42.799513 29568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:42.800704 29568 hybrid_clock.cc:648] HybridClock initialized: now 1786084842800671 us; error 47 us; skew 500 ppm
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:42.804193 29568 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:40:42.804797 29568 webserver.cc:460] Webserver started at http://127.11.114.193:46003/ using document root <none> and password file <none>
I20260807 06:40:42.805078 29568 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:42.805169 29568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:42.831691 29568 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:42.832834 29643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.833057 29568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:42.833186 29568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "88b2142f44d94ed8bf006036815eaab6"
format_stamp: "Formatted at 2026-08-07 06:40:41 on dist-test-slave-gkw7"
server_key: "5bf12f7bda8e264747de9649912495c5"
server_key_iv: "d8aea84983c2dfd65a71c6479d01e1d9"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:42.833616 29568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:42.845324 29568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:42.847918 29568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:42.848081 29568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:42.848358 29568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:42.848759 29568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:42.848802 29568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.848832 29568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:42.848850 29568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:42.855770 29568 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:36323
I20260807 06:40:42.856243 29568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:40:42.856873 29829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:36323 every 8 connection(s)
I20260807 06:40:42.862445 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 29568
I20260807 06:40:42.862798 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:52271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:36679
--local_ip_for_outbound_sockets=127.11.114.194
--tserver_master_addrs=127.11.114.254:39213
--webserver_port=45423
--webserver_interface=127.11.114.194
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.114.212:52271/kms with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:40:42 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084842, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:42.900481 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:42.858148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:36625 (local address 127.11.114.254:39213)
0807 06:40:42.868304 (+ 10156us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:42.868317 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:42.868401 (+ 84us) server_negotiation.cc:407] Connection header received
0807 06:40:42.868711 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:42.868722 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:42.868826 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:42.869050 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:42.870054 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.872957 (+ 2903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.878200 (+ 5243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.879739 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.881216 (+ 1477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:42.891997 (+ 10781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:42.892002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:42.892002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:42.892017 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:42.895439 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:42.898508 (+ 3069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:42.898513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:42.898513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:42.898653 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:42.898956 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:42.898956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:42.898957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:42.899256 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:42.899402 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:42.900161 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 06:40:42.900290 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10044,"thread_start_us":84,"threads_started":1}
I20260807 06:40:42.901105 29836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:42.857693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39213 (local address 127.11.114.193:36625)
0807 06:40:42.865794 (+ 8101us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:42.865868 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:42.866499 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:42.869245 (+ 2746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:42.869250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:42.869523 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:42.869891 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:42.869900 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.873099 (+ 3199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.873100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:42.878004 (+ 4904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:42.878009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:42.881348 (+ 3339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:42.881349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:42.881494 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:42.882187 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:42.882198 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:42.891830 (+ 9632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:42.895554 (+ 3724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:42.895565 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:42.895567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:42.895907 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:42.898738 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:42.898739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:42.898739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:42.898888 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:42.899423 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:42.899429 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:42.899578 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 06:40:42.899683 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 06:40:42.899853 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8033,"thread_start_us":112,"threads_started":1}
I20260807 06:40:42.902189 29833 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39213
I20260807 06:40:42.902287 29833 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:42.902490 29833 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
W20260807 06:40:42.903666 29833 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:39213 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 06:40:42.903734 29833 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:42.903795 29833 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
I20260807 06:40:42.904274 29833 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:42.904352 29833 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
W20260807 06:40:42.904734 29833 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:40:42.987712 29840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:42.988042 29840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:42.988085 29840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:42.989935 29840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:42.989996 29840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:42.990042 29840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:40:42.992358 29840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.114.212:52271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:36679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=45423
--tserver_master_addrs=127.11.114.254:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:42.992654 29840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:42.992941 29840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:43.000152 29852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:43.000152 29854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:43.000523 29848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:43.001413 29840 server_base.cc:1034] running on GCE node
I20260807 06:40:43.001658 29840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:43.001976 29840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:43.005230 29840 hybrid_clock.cc:648] HybridClock initialized: now 1786084843005190 us; error 50 us; skew 500 ppm
Aug 07 06:40:43 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084843, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:43.008208 29840 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:40:43.008797 29840 webserver.cc:460] Webserver started at http://127.11.114.194:45423/ using document root <none> and password file <none>
I20260807 06:40:43.009122 29840 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:43.009217 29840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:40:43 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084843, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:43.025821 29840 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:43.026780 29864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:43.026968 29840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:43.027060 29840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "3d882df0fc0747d0986c6aab9aa7ee29"
format_stamp: "Formatted at 2026-08-07 06:40:42 on dist-test-slave-gkw7"
server_key: "00533cb6c648efca287a8ecf23d8dd34"
server_key_iv: "3888ee7d99104b1f256dd7f4954d8196"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:43.028337 29840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:43.040547 29840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:43.043135 29840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:43.043329 29840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:43.043552 29840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:43.043951 29840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:43.044001 29840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:43.044034 29840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:43.044050 29840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:43.051290 29840 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:36679
I20260807 06:40:43.051690 30038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:36679 every 8 connection(s)
I20260807 06:40:43.051844 29840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:40:43.060813 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 29840
I20260807 06:40:43.061182 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.114.212:52271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:39633
--local_ip_for_outbound_sockets=127.11.114.195
--tserver_master_addrs=127.11.114.254:39213
--webserver_port=40119
--webserver_interface=127.11.114.195
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:44209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.114.212:52271/kms with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:40:43 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084843, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:43.073505 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:43.052990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:33759 (local address 127.11.114.254:39213)
0807 06:40:43.053038 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:43.053041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:43.053052 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:40:43.055601 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:43.055603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:43.055619 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:43.055693 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:43.064268 (+ 8575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.065009 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.066586 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.066852 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.066903 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:43.069967 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:43.069970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:43.069970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:43.069978 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:43.071825 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:43.072344 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:43.072345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:43.072346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:43.072407 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:43.072662 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:43.072663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:43.072663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:43.072795 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:43.072844 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:43.073250 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 06:40:43.073362 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:40:43.073594 30042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:43.052695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39213 (local address 127.11.114.194:33759)
0807 06:40:43.052905 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:43.052959 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:43.055433 (+ 2474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:43.055781 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:43.055787 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:43.056052 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:43.063999 (+ 7947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:43.064014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.065172 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.065173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:43.066468 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:43.066472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.067029 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.067030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:43.067199 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:43.068049 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:43.068060 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:43.069844 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:43.071922 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:43.071929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:43.071932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:43.072252 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:43.072470 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:43.072471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:43.072471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:43.072595 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:43.072876 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:43.072882 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:43.073049 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 06:40:43.073153 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 06:40:43.073329 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260807 06:40:43.074462 30039 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39213
I20260807 06:40:43.074572 30039 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:43.074808 30039 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
W20260807 06:40:43.075582 30039 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:39213 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 06:40:43.075673 30039 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:43.075748 30039 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
I20260807 06:40:43.076083 30039 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:43.076155 30039 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
W20260807 06:40:43.076463 30039 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:40:43.172760 30044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:43.173049 30044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:43.173102 30044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:43.174808 30044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:43.174880 30044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:43.174930 30044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:40:43.177146 30044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:44209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.114.212:52271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:39633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=40119
--tserver_master_addrs=127.11.114.254:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:43.177450 30044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:43.177768 30044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:43.184862 30050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:43.185230 30053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:43.186636 30044 server_base.cc:1034] running on GCE node
W20260807 06:40:43.188783 30049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:43.189110 30044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:43.189426 30044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:43.190601 30044 hybrid_clock.cc:648] HybridClock initialized: now 1786084843190564 us; error 53 us; skew 500 ppm
Aug 07 06:40:43 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084843, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:43.194031 30044 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:40:43.194599 30044 webserver.cc:460] Webserver started at http://127.11.114.195:40119/ using document root <none> and password file <none>
I20260807 06:40:43.194885 30044 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:43.194957 30044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:40:43 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084843, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:43.231630 30044 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 06:40:43.232627 30064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:43.232892 30044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:40:43.232990 30044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "a0b928d793f24b69a78dc3d2e095c8ac"
format_stamp: "Formatted at 2026-08-07 06:40:42 on dist-test-slave-gkw7"
server_key: "d5786eeefb7da48bdcd6178bd4a2dcf0"
server_key_iv: "f3f50f8d51e32091674ecbeca21a9bbd"
server_key_version: "kuduclusterkey@0"
I20260807 06:40:43.234227 30044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:43.265499 30044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:43.268090 30044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:43.268278 30044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:43.268610 30044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:43.269016 30044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:43.269073 30044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:43.269114 30044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:43.269146 30044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:43.276783 30044 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:39633
I20260807 06:40:43.277084 30246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:39633 every 8 connection(s)
I20260807 06:40:43.277300 30044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:40:43.285346 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 30044
Aug 07 06:40:43 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084843, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:43.296950 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:43.280666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:47035 (local address 127.11.114.254:39213)
0807 06:40:43.280721 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:43.280724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:43.281111 (+ 387us) server_negotiation.cc:407] Connection header received
0807 06:40:43.281806 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:43.281807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:43.281822 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:43.281856 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:43.282742 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.283383 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.284891 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.285212 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.285262 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:43.292730 (+ 7468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:43.292731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:43.292732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:43.292740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:43.294924 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:43.295514 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:43.295515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:43.295516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:43.295583 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:43.295906 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:43.295907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:43.295907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:43.296055 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:43.296138 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:43.296743 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 06:40:43.296838 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:40:43.297628 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:43.280694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39213 (local address 127.11.114.195:47035)
0807 06:40:43.280974 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:43.281018 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:43.281656 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:43.282055 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:43.282059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:43.282324 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:43.282620 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:43.282627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.283501 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.283502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:43.284782 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:43.284784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.285388 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.285389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:43.285574 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:43.286841 (+ 1267us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:43.286851 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:43.289840 (+ 2989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:43.295044 (+ 5204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:43.295053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:43.295055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:43.295425 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:43.295661 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:43.295662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:43.295662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:43.295820 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:43.296170 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:43.296178 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:43.296367 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 06:40:43.296484 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 06:40:43.296652 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":205,"threads_started":1}
I20260807 06:40:43.298332 30248 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39213
I20260807 06:40:43.298386 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:43.285791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.11.114.254:39213)
0807 06:40:43.285902 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:43.285904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:43.286310 (+ 406us) server_negotiation.cc:407] Connection header received
0807 06:40:43.288114 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:43.288115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:43.288132 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:43.288207 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:43.290731 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.291484 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.292612 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:43.292912 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:43.292978 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:43.295057 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:43.295058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:43.295059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:43.295065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:43.296791 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:43.297187 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:43.297188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:43.297188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:43.297245 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:43.297425 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:43.297426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:43.297426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:43.297573 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:43.297752 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:43.298194 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 06:40:43.298287 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":65,"threads_started":1}
I20260807 06:40:43.298426 30248 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:43.298610 30248 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
W20260807 06:40:43.299324 30248 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:39213 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 06:40:43.299417 30248 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:43.299491 30248 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
I20260807 06:40:43.299925 30248 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:43.300004 30248 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
W20260807 06:40:43.300298 30248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 06:40:43 dist-test-slave-gkw7 krb5kdc[28123](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084843, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:43.904928 29833 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:43.905135 29833 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
I20260807 06:40:43.905589 29515 ts_manager.cc:194] Registered new tserver with Master: 88b2142f44d94ed8bf006036815eaab6 (127.11.114.193:36323)
I20260807 06:40:43.906632 29515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:36625
I20260807 06:40:44.076658 30039 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:44.076831 30039 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
I20260807 06:40:44.077203 29515 ts_manager.cc:194] Registered new tserver with Master: 3d882df0fc0747d0986c6aab9aa7ee29 (127.11.114.194:36679)
I20260807 06:40:44.077790 29515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:33759
Aug 07 06:40:44 dist-test-slave-gkw7 krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084843, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for HTTP/127.11.114.212@KRBTEST.COM
I20260807 06:40:44.300513 30248 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:44.300691 30248 heartbeater.cc:511] Master 127.11.114.254:39213 requested a full tablet report, sending...
I20260807 06:40:44.301013 29515 ts_manager.cc:194] Registered new tserver with Master: a0b928d793f24b69a78dc3d2e095c8ac (127.11.114.195:39633)
I20260807 06:40:44.301606 29515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:47035
I20260807 06:40:44.305827 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:40:44.318189 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.307635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41680 (local address 127.11.114.254:39213)
0807 06:40:44.309813 (+ 2178us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:44.309815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:44.309823 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:40:44.309845 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:44.309846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:44.309858 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:44.309916 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:44.310636 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.311423 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.312462 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.312795 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.312858 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.314829 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:44.314831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:44.314831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:44.314838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:44.316670 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:44.317048 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:44.317049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:44.317049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:44.317108 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:44.317292 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:44.317293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:44.317293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:44.317415 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:44.317503 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:44.317999 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 06:40:44.318085 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2157,"thread_start_us":68,"threads_started":1}
I20260807 06:40:44.320458 29515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:40:44.322722 29515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:40:44.342926 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.332035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:36679 (local address 127.0.0.1:57974)
0807 06:40:44.337613 (+ 5578us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:44.337626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:44.337703 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:44.337973 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:44.337976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:44.337979 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:44.338064 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.338066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.339842 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.339843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.341232 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.341235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.342334 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.342335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.342420 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.342434 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 06:40:44.342501 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:40:44.342832 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5557,"thread_start_us":74,"threads_started":1}
I20260807 06:40:44.343286 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.332492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.11.114.194:36679)
0807 06:40:44.336587 (+ 4095us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:44.336593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:44.337722 (+ 1129us) server_negotiation.cc:407] Connection header received
0807 06:40:44.337787 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:44.337790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:44.337808 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:44.337883 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:44.338210 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.339757 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.341335 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.342163 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.342239 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.342297 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:44.343165 (+ 868us) server_negotiation.cc:299] Negotiation successful
0807 06:40:44.343208 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4053,"thread_start_us":57,"threads_started":1}
I20260807 06:40:44.344152 29936 tablet_service.cc:1467] Processing CreateTablet for tablet 017b6e5047e44819a76c5d3812c17e9e (DEFAULT_TABLE table=test-table [id=4e7c99e68e104674a710c3fd5c7042b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:44.344542 29936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017b6e5047e44819a76c5d3812c17e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:44.348718 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.331881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:36323 (local address 127.0.0.1:37638)
0807 06:40:44.333863 (+ 1982us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:44.333870 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:44.333903 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:44.344110 (+ 10207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:44.344111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:44.344114 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:44.344160 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.344162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.345996 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.345997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.347320 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.347323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.348438 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.348439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.348509 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.348513 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:40:44.348567 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 06:40:44.348630 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1969,"thread_start_us":59,"threads_started":1}
I20260807 06:40:44.350270 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.332333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37638 (local address 127.11.114.193:36323)
0807 06:40:44.343734 (+ 11401us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:44.343740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:44.343755 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:40:44.343799 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:44.343802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:44.343820 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:44.343919 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:44.344254 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.345426 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.347432 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.348258 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.348344 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.348398 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:44.350113 (+ 1715us) server_negotiation.cc:299] Negotiation successful
0807 06:40:44.350176 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11359,"thread_start_us":57,"threads_started":1}
I20260807 06:40:44.351058 29707 tablet_service.cc:1467] Processing CreateTablet for tablet 017b6e5047e44819a76c5d3812c17e9e (DEFAULT_TABLE table=test-table [id=4e7c99e68e104674a710c3fd5c7042b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:44.351459 29707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017b6e5047e44819a76c5d3812c17e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:44.358419 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.332414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.11.114.195:39633)
0807 06:40:44.345613 (+ 13199us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:44.345618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:44.345630 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:40:44.345667 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:44.345669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:44.345687 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:44.345774 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:44.348849 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.349967 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.357028 (+ 7061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.357918 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.357984 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.358037 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:44.358280 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 06:40:44.358327 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13158,"thread_start_us":71,"threads_started":1}
I20260807 06:40:44.358695 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.331680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:39633 (local address 127.0.0.1:37484)
0807 06:40:44.333646 (+ 1966us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:44.333690 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:44.333760 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:44.345851 (+ 12091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:44.345852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:44.345855 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:44.345895 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.345897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.350068 (+ 4171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.350069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.356904 (+ 6835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.356908 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.358107 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.358108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.358181 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.358185 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:40:44.358550 (+ 365us) client_negotiation.cc:240] Negotiation successful
0807 06:40:44.358613 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1929,"thread_start_us":74,"threads_started":1}
I20260807 06:40:44.359417 30139 tablet_service.cc:1467] Processing CreateTablet for tablet 017b6e5047e44819a76c5d3812c17e9e (DEFAULT_TABLE table=test-table [id=4e7c99e68e104674a710c3fd5c7042b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:44.359788 30139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017b6e5047e44819a76c5d3812c17e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:44.359766 30295 tablet_bootstrap.cc:492] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: Bootstrap starting.
I20260807 06:40:44.360724 30295 tablet_bootstrap.cc:654] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:44.361968 30295 log.cc:824] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:44.365960 30296 tablet_bootstrap.cc:492] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6: Bootstrap starting.
I20260807 06:40:44.366775 30296 tablet_bootstrap.cc:654] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:44.367179 30296 log.cc:824] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:44.367537 30297 tablet_bootstrap.cc:492] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: Bootstrap starting.
I20260807 06:40:44.368255 30297 tablet_bootstrap.cc:654] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:44.368716 30297 log.cc:824] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:44.370030 30295 tablet_bootstrap.cc:492] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: No bootstrap required, opened a new log
I20260807 06:40:44.370177 30295 ts_tablet_manager.cc:1397] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260807 06:40:44.373040 30295 raft_consensus.cc:348] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.373332 30295 raft_consensus.cc:374] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:44.373417 30295 raft_consensus.cc:729] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d882df0fc0747d0986c6aab9aa7ee29, State: Initialized, Role: FOLLOWER
I20260807 06:40:44.373641 30295 consensus_queue.cc:260] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.374014 30295 ts_tablet_manager.cc:1428] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 06:40:44.374193 30039 heartbeater.cc:503] Master 127.11.114.254:39213 was elected leader, sending a full tablet report...
I20260807 06:40:44.377985 30296 tablet_bootstrap.cc:492] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6: No bootstrap required, opened a new log
I20260807 06:40:44.378134 30296 ts_tablet_manager.cc:1397] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260807 06:40:44.380882 30296 raft_consensus.cc:348] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.381549 30297 tablet_bootstrap.cc:492] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: No bootstrap required, opened a new log
I20260807 06:40:44.381677 30297 ts_tablet_manager.cc:1397] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260807 06:40:44.384539 30297 raft_consensus.cc:348] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.384764 30296 raft_consensus.cc:374] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:44.384872 30296 raft_consensus.cc:729] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88b2142f44d94ed8bf006036815eaab6, State: Initialized, Role: FOLLOWER
I20260807 06:40:44.385051 30296 consensus_queue.cc:260] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.385425 30296 ts_tablet_manager.cc:1428] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260807 06:40:44.385603 29833 heartbeater.cc:503] Master 127.11.114.254:39213 was elected leader, sending a full tablet report...
I20260807 06:40:44.385919 30297 raft_consensus.cc:374] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:44.386016 30297 raft_consensus.cc:729] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b928d793f24b69a78dc3d2e095c8ac, State: Initialized, Role: FOLLOWER
I20260807 06:40:44.386198 30297 consensus_queue.cc:260] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.386518 30297 ts_tablet_manager.cc:1428] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 06:40:44.386663 30248 heartbeater.cc:503] Master 127.11.114.254:39213 was elected leader, sending a full tablet report...
I20260807 06:40:44.408164 30301 raft_consensus.cc:482] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:40:44.408318 30301 raft_consensus.cc:504] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.408740 30301 leader_election.cc:288] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0b928d793f24b69a78dc3d2e095c8ac (127.11.114.195:39633), 88b2142f44d94ed8bf006036815eaab6 (127.11.114.193:36323)
I20260807 06:40:44.413236 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.408981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:45707 (local address 127.11.114.195:39633)
0807 06:40:44.409045 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:44.409048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:44.409229 (+ 181us) server_negotiation.cc:407] Connection header received
0807 06:40:44.409346 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:44.409348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:44.409361 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:44.409394 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:44.409807 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.410533 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.412153 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.412764 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.412815 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.412826 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:44.413115 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 06:40:44.413160 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 06:40:44.413714 30304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.408927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:39633 (local address 127.11.114.194:45707)
0807 06:40:44.409154 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:44.409166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:44.409261 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:44.409469 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:44.409470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:44.409472 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:44.409716 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.409718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.410641 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.410642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.412043 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.412046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.412930 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.412931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.413010 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.413013 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:40:44.413568 (+ 555us) client_negotiation.cc:240] Negotiation successful
0807 06:40:44.413616 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":78,"thread_start_us":113,"threads_started":1}
I20260807 06:40:44.414103 30175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017b6e5047e44819a76c5d3812c17e9e" candidate_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" is_pre_election: true
I20260807 06:40:44.414296 30175 raft_consensus.cc:2454] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d882df0fc0747d0986c6aab9aa7ee29 in term 0.
I20260807 06:40:44.414546 29869 leader_election.cc:302] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d882df0fc0747d0986c6aab9aa7ee29, a0b928d793f24b69a78dc3d2e095c8ac; no voters:
I20260807 06:40:44.414700 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.409340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:34409 (local address 127.11.114.193:36323)
0807 06:40:44.409391 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:44.409393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:44.409404 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:40:44.409444 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:44.409445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:44.409455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:44.409577 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:44.409774 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.410535 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.412153 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.412808 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.412850 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.412858 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:44.414588 (+ 1730us) server_negotiation.cc:299] Negotiation successful
0807 06:40:44.414636 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 06:40:44.414709 30301 raft_consensus.cc:2781] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:40:44.414754 30301 raft_consensus.cc:482] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:40:44.414777 30301 raft_consensus.cc:3037] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:44.414955 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.409159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:36323 (local address 127.11.114.194:34409)
0807 06:40:44.409344 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:44.409354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:44.409395 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:44.409570 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:44.409572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:44.409573 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:44.409639 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.409642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.410640 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.410642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.412043 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:44.412046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.412959 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.412960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:44.413029 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.413032 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:40:44.414780 (+ 1748us) client_negotiation.cc:240] Negotiation successful
0807 06:40:44.414827 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":73,"thread_start_us":71,"threads_started":1}
I20260807 06:40:44.415731 30301 raft_consensus.cc:504] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.415881 30301 leader_election.cc:288] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [CANDIDATE]: Term 1 election: Requested vote from peers a0b928d793f24b69a78dc3d2e095c8ac (127.11.114.195:39633), 88b2142f44d94ed8bf006036815eaab6 (127.11.114.193:36323)
I20260807 06:40:44.415894 29743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017b6e5047e44819a76c5d3812c17e9e" candidate_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88b2142f44d94ed8bf006036815eaab6" is_pre_election: true
I20260807 06:40:44.416047 29743 raft_consensus.cc:2454] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d882df0fc0747d0986c6aab9aa7ee29 in term 0.
I20260807 06:40:44.416116 30175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017b6e5047e44819a76c5d3812c17e9e" candidate_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0b928d793f24b69a78dc3d2e095c8ac"
I20260807 06:40:44.416189 30175 raft_consensus.cc:3037] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:44.416199 29742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017b6e5047e44819a76c5d3812c17e9e" candidate_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88b2142f44d94ed8bf006036815eaab6"
I20260807 06:40:44.416249 29742 raft_consensus.cc:3037] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:44.416956 29742 raft_consensus.cc:2454] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d882df0fc0747d0986c6aab9aa7ee29 in term 1.
I20260807 06:40:44.417079 30175 raft_consensus.cc:2454] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d882df0fc0747d0986c6aab9aa7ee29 in term 1.
I20260807 06:40:44.417207 29869 leader_election.cc:302] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d882df0fc0747d0986c6aab9aa7ee29, a0b928d793f24b69a78dc3d2e095c8ac; no voters:
I20260807 06:40:44.417311 30301 raft_consensus.cc:2781] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:40:44.417500 30301 raft_consensus.cc:686] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 1 LEADER]: Becoming Leader. State: Replica: 3d882df0fc0747d0986c6aab9aa7ee29, State: Running, Role: LEADER
I20260807 06:40:44.417645 30301 consensus_queue.cc:237] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } }
I20260807 06:40:44.418457 29515 catalog_manager.cc:5495] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d882df0fc0747d0986c6aab9aa7ee29 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d882df0fc0747d0986c6aab9aa7ee29" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 36679 } health_report { overall_health: HEALTHY } } }
I20260807 06:40:44.440671 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.436628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.11.114.194:36679)
0807 06:40:44.436671 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:44.436674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:44.436682 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:40:44.436725 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:44.436726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:44.436748 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:44.436838 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:40:44.437053 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.438055 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.439417 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:44.439777 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:44.439844 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:44.440056 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:40:44.440201 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:40:44.440283 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:44.440543 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 06:40:44.440566 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:40:44.442734 29742 raft_consensus.cc:1261] T 017b6e5047e44819a76c5d3812c17e9e P 88b2142f44d94ed8bf006036815eaab6 [term 1 FOLLOWER]: Refusing update from remote peer 3d882df0fc0747d0986c6aab9aa7ee29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:40:44.442761 30175 raft_consensus.cc:1261] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 1 FOLLOWER]: Refusing update from remote peer 3d882df0fc0747d0986c6aab9aa7ee29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:40:44.443208 30306 consensus_queue.cc:1035] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88b2142f44d94ed8bf006036815eaab6" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 36323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:40:44.443406 30301 consensus_queue.cc:1035] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0b928d793f24b69a78dc3d2e095c8ac" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 39633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:40:44.447209 30309 mvcc.cc:205] Tried to move back new op lower bound from 7315803522834300928 to 7315803522735243264. Current Snapshot: MvccSnapshot[applied={T|T < 7315803522834300928 or (T in {7315803522834300928})}]
I20260807 06:40:44.447329 30311 mvcc.cc:205] Tried to move back new op lower bound from 7315803522834300928 to 7315803522735243264. Current Snapshot: MvccSnapshot[applied={T|T < 7315803522834300928}]
I20260807 06:40:44.449128 30310 mvcc.cc:205] Tried to move back new op lower bound from 7315803522834300928 to 7315803522735243264. Current Snapshot: MvccSnapshot[applied={T|T < 7315803522834300928}]
I20260807 06:40:44.449193 29514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:40:44.449287 29514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:40:44.450112 29514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd6aeb6192bb48b49eb69a9d7653d105: Sending DeleteTablet for 3 replicas of tablet 017b6e5047e44819a76c5d3812c17e9e
I20260807 06:40:44.450465 29936 tablet_service.cc:1514] Processing DeleteTablet for tablet 017b6e5047e44819a76c5d3812c17e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:44 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:57974
I20260807 06:40:44.451066 30139 tablet_service.cc:1514] Processing DeleteTablet for tablet 017b6e5047e44819a76c5d3812c17e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:44 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:37484
I20260807 06:40:44.451342 29707 tablet_service.cc:1514] Processing DeleteTablet for tablet 017b6e5047e44819a76c5d3812c17e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:44 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:37638
I20260807 06:40:44.452178 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 29568
I20260807 06:40:44.465348 30329 tablet_replica.cc:330] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: stopping tablet replica
I20260807 06:40:44.465566 30329 raft_consensus.cc:2229] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:40:44.465801 30329 raft_consensus.cc:2258] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:40:44.466034 30327 tablet_replica.cc:330] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: stopping tablet replica
I20260807 06:40:44.466560 30329 ts_tablet_manager.cc:1905] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:40:44.466742 30327 raft_consensus.cc:2229] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:40:44.466925 30327 raft_consensus.cc:2258] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:40:44.468792 30329 ts_tablet_manager.cc:1918] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:40:44.469292 30327 ts_tablet_manager.cc:1905] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:40:44.470966 30327 ts_tablet_manager.cc:1918] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:40:44.471119 30329 log.cc:1196] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/017b6e5047e44819a76c5d3812c17e9e
I20260807 06:40:44.471496 30329 ts_tablet_manager.cc:1939] T 017b6e5047e44819a76c5d3812c17e9e P a0b928d793f24b69a78dc3d2e095c8ac: Deleting consensus metadata
I20260807 06:40:44.471624 30327 log.cc:1196] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/017b6e5047e44819a76c5d3812c17e9e
I20260807 06:40:44.472173 29497 catalog_manager.cc:4841] TS a0b928d793f24b69a78dc3d2e095c8ac (127.11.114.195:39633): tablet 017b6e5047e44819a76c5d3812c17e9e (table test-table [id=4e7c99e68e104674a710c3fd5c7042b8]) successfully deleted
I20260807 06:40:44.472606 30327 ts_tablet_manager.cc:1939] T 017b6e5047e44819a76c5d3812c17e9e P 3d882df0fc0747d0986c6aab9aa7ee29: Deleting consensus metadata
I20260807 06:40:44.473104 29491 catalog_manager.cc:4841] TS 3d882df0fc0747d0986c6aab9aa7ee29 (127.11.114.194:36679): tablet 017b6e5047e44819a76c5d3812c17e9e (table test-table [id=4e7c99e68e104674a710c3fd5c7042b8]) successfully deleted
W20260807 06:40:44.473385 29497 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:36323 (error 108)
I20260807 06:40:44.474141 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 29840
I20260807 06:40:44.478089 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:44.477953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:36323 (local address 127.0.0.1:37642)
0807 06:40:44.478010 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:44.478050 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:36323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 06:40:44.478295 29497 catalog_manager.cc:4584] TS 88b2142f44d94ed8bf006036815eaab6 (127.11.114.193:36323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 017b6e5047e44819a76c5d3812c17e9e: Network error: Client connection negotiation failed: client connection to 127.11.114.193:36323: connect: Connection refused (error 111)
I20260807 06:40:44.483006 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 30044
I20260807 06:40:44.495450 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 29464
I20260807 06:40:44.507654 11723 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 06:40:44.852393 11723 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 06:40:44.852511 11723 mini_ranger.cc:66] Stopping Ranger...
I20260807 06:40:45.197671 11723 mini_ranger.cc:68] Stopped Ranger
2026-08-07 06:40:45.197 UTC [28165] LOG: received smart shutdown request
2026-08-07 06:40:45.202 UTC [28165] LOG: background worker "logical replication launcher" (PID 28175) exited with exit code 1
2026-08-07 06:40:45.202 UTC [28170] LOG: shutting down
2026-08-07 06:40:45.395 UTC [28165] LOG: database system is shut down
2026-08-07T06:40:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51167 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:40:45 dist-test-slave-gkw7 krb5kdc[30348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:40:45 dist-test-slave-gkw7 krb5kdc[30348](info): set up 2 sockets
Aug 07 06:40:45 dist-test-slave-gkw7 krb5kdc[30348](info): commencing operation
krb5kdc: starting...
W20260807 06:40:46.727543 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:40:46 dist-test-slave-gkw7 krb5kdc[30348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:40:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:40:46.854586 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:35025
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:35025,127.11.114.253:33533,127.11.114.252:39131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:46.937254 30368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:46.937516 30368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:46.937551 30368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:46.938715 30368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:46.938793 30368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:46.938817 30368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:46.938835 30368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:46.938861 30368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:46.940382 30368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:35025,127.11.114.253:33533,127.11.114.252:39131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:35025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:46.940680 30368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:46.940918 30368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:46.945982 30374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:46.945989 30375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:46.946026 30377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:46.946341 30368 server_base.cc:1034] running on GCE node
I20260807 06:40:46.946678 30368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:46.947018 30368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:46.948226 30368 hybrid_clock.cc:648] HybridClock initialized: now 1786084846948205 us; error 44 us; skew 500 ppm
Aug 07 06:40:46 dist-test-slave-gkw7 krb5kdc[30348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084846, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:46.954262 30368 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:40:46.954778 30368 webserver.cc:460] Webserver started at http://127.11.114.254:39193/ using document root <none> and password file <none>
I20260807 06:40:46.955017 30368 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:46.955088 30368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:46.955246 30368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:40:46.956352 30368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "d4315940690b465ba0beb541c2b85b82"
format_stamp: "Formatted at 2026-08-07 06:40:46 on dist-test-slave-gkw7"
server_key: "615a26ff31767bc30b7df6090c67d5e6"
server_key_iv: "caff4e5439f95d3cdeea24bbc4297c4a"
server_key_version: "encryptionkey@0"
I20260807 06:40:46.956746 30368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "d4315940690b465ba0beb541c2b85b82"
format_stamp: "Formatted at 2026-08-07 06:40:46 on dist-test-slave-gkw7"
server_key: "615a26ff31767bc30b7df6090c67d5e6"
server_key_iv: "caff4e5439f95d3cdeea24bbc4297c4a"
server_key_version: "encryptionkey@0"
I20260807 06:40:46.958276 30368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:40:46.959069 30387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:46.959313 30368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:46.959431 30368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "d4315940690b465ba0beb541c2b85b82"
format_stamp: "Formatted at 2026-08-07 06:40:46 on dist-test-slave-gkw7"
server_key: "615a26ff31767bc30b7df6090c67d5e6"
server_key_iv: "caff4e5439f95d3cdeea24bbc4297c4a"
server_key_version: "encryptionkey@0"
I20260807 06:40:46.959515 30368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:46.979254 30368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:46.981630 30368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:46.981834 30368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:46.986157 30368 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:35025
I20260807 06:40:46.986195 30466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:35025 every 8 connection(s)
I20260807 06:40:46.986621 30368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:40:46.987591 30469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:46.989243 30469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:40:46.989892 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 30368
I20260807 06:40:46.990038 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:40:46.990289 11723 external_mini_cluster.cc:1428] Setting key 4b700cd51b5c51e92157dc23264dffcc
Aug 07 06:40:46 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084846, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.253@KRBTEST.COM; defaulting to no policy
I20260807 06:40:47.000515 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:46.990089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42600 (local address 127.11.114.254:35025)
0807 06:40:46.990289 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:46.990301 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:46.990448 (+ 147us) server_negotiation.cc:407] Connection header received
0807 06:40:46.991035 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:46.991037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:46.991060 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:46.991162 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:46.991812 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:46.992691 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:46.993710 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:46.994256 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:46.994386 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:46.996819 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:46.996828 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:46.996829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:46.996838 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:46.998785 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:46.999241 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:46.999245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:46.999245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:46.999329 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:46.999565 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:46.999565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:46.999566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:46.999877 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:46.999944 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.000206 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.000317 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260807 06:40:47.000519 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:46.990072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35025 (local address 127.0.0.1:42600)
0807 06:40:46.990310 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:46.990352 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:46.990863 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:46.991168 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:46.991171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:46.991475 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:46.991666 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:46.991673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:46.992798 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:46.992799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:46.993628 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:46.993630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:46.994385 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:46.994386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:46.994488 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:46.995165 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:46.995175 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:46.996722 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:46.998914 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:46.998922 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:46.998925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:46.999157 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:46.999402 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:46.999402 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:46.999403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:46.999490 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:46.999971 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:46.999979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.000071 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.000163 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.000385 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":77,"thread_start_us":104,"threads_started":1}
I20260807 06:40:47.001389 30469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.253@KRBTEST.COM" created.
I20260807 06:40:47.001917 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.001758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54726)
0807 06:40:47.001815 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.001888 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 06:40:47.002064 30393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
I20260807 06:40:47.002583 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.002466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54736)
0807 06:40:47.002517 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.002550 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 06:40:47.002739 30469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:33533: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab.
Entry for principal kudu/127.11.114.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab.
I20260807 06:40:47.018868 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.253:33533
--webserver_interface=127.11.114.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab
--principal=kudu/127.11.114.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:35025,127.11.114.253:33533,127.11.114.252:39131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:40:47.052006 30469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } attempt: 1
I20260807 06:40:47.052595 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.052440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54750)
0807 06:40:47.052516 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.052561 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 06:40:47.052960 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.052861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54766)
0807 06:40:47.052909 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.052937 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 06:40:47.053054 30469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:33533: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
W20260807 06:40:47.103659 30477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:47.103902 30477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:47.103938 30477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:47.105067 30477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:47.105134 30477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:47.105165 30477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:47.105191 30477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:47.105223 30477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:47.106755 30477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:35025,127.11.114.253:33533,127.11.114.252:39131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab
--principal=kudu/127.11.114.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.253:33533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.114.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:47.107031 30477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:47.107264 30477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:47.112430 30488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:47.112519 30493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:47.112743 30477 server_base.cc:1034] running on GCE node
W20260807 06:40:47.112792 30489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:47.113108 30477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:47.113415 30477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:47.114637 30477 hybrid_clock.cc:648] HybridClock initialized: now 1786084847114610 us; error 41 us; skew 500 ppm
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084847, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:47.117400 30477 init.cc:377] Logged in from keytab as kudu/127.11.114.253@KRBTEST.COM (short username kudu)
I20260807 06:40:47.117904 30477 webserver.cc:460] Webserver started at http://127.11.114.253:35375/ using document root <none> and password file <none>
I20260807 06:40:47.118147 30477 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:47.118193 30477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:47.118348 30477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:40:47.119450 30477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/data/instance:
uuid: "09215efa3c7a408aa85e2de4cb7f7c92"
format_stamp: "Formatted at 2026-08-07 06:40:47 on dist-test-slave-gkw7"
server_key: "629ec7b29e0aa6360a0f04f4625add71"
server_key_iv: "99720b160d1afee4040a435895c7a335"
server_key_version: "encryptionkey@0"
I20260807 06:40:47.119935 30477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/wal/instance:
uuid: "09215efa3c7a408aa85e2de4cb7f7c92"
format_stamp: "Formatted at 2026-08-07 06:40:47 on dist-test-slave-gkw7"
server_key: "629ec7b29e0aa6360a0f04f4625add71"
server_key_iv: "99720b160d1afee4040a435895c7a335"
server_key_version: "encryptionkey@0"
I20260807 06:40:47.121176 30469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } attempt: 2
I20260807 06:40:47.121439 30477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:40:47.121629 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.121473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54774)
0807 06:40:47.121558 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.121595 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 06:40:47.121963 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.121874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54778)
0807 06:40:47.121919 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.121942 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 06:40:47.122058 30469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:33533: Network error: Client connection negotiation failed: client connection to 127.11.114.253:33533: connect: Connection refused (error 111)
I20260807 06:40:47.122398 30509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:47.122571 30477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:47.122696 30477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/wal
uuid: "09215efa3c7a408aa85e2de4cb7f7c92"
format_stamp: "Formatted at 2026-08-07 06:40:47 on dist-test-slave-gkw7"
server_key: "629ec7b29e0aa6360a0f04f4625add71"
server_key_iv: "99720b160d1afee4040a435895c7a335"
server_key_version: "encryptionkey@0"
I20260807 06:40:47.122776 30477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:47.134667 30477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:47.136696 30477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:47.136862 30477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:47.141013 30477 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.253:33533
I20260807 06:40:47.141052 30577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.253:33533 every 8 connection(s)
I20260807 06:40:47.141443 30477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
I20260807 06:40:47.142444 30578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:47.143989 30578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:40:47.144963 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 30477
I20260807 06:40:47.145098 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-1/wal/instance
I20260807 06:40:47.145310 11723 external_mini_cluster.cc:1428] Setting key 48b4ed98b4208c1c20252ede4870f75b
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084847, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.252@KRBTEST.COM; defaulting to no policy
I20260807 06:40:47.154940 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.144724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42610 (local address 127.11.114.254:35025)
0807 06:40:47.144796 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.144799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.145032 (+ 233us) server_negotiation.cc:407] Connection header received
0807 06:40:47.145705 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.145706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.145723 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.145800 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.146388 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.147159 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.148638 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.149009 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.149089 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.151621 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.151623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.151624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.151633 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.153373 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.153850 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.153852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.153852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.153921 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.154212 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.154214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.154214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.154359 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.154424 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.154743 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.154843 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:40:47.155141 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.144710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35025 (local address 127.0.0.1:42610)
0807 06:40:47.144886 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.144926 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:47.145520 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:47.145851 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:47.145854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:47.146109 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:47.146300 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.146304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.147262 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.147264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.148550 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.148553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.149134 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.149136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.149252 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.149916 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:47.149928 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:47.151523 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:47.153466 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.153472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.153474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.153773 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.154015 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.154015 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.154016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.154128 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.154439 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:47.154446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.154566 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.154700 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.154865 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260807 06:40:47.155773 30578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.252@KRBTEST.COM" created.
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084847, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:40:47.164152 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.156081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54790)
0807 06:40:47.156130 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.156141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:47.156198 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:47.156438 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:47.156439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:47.156579 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:47.156631 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.156632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.157687 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.157687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.158192 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.158194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.158862 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.158863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.158949 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.159398 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:47.159404 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:47.160734 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:47.163032 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.163033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.163033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.163260 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.163493 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.163493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.163494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.163548 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.163906 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:47.163907 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.163946 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.164005 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.164082 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:40:47.164295 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.156130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.11.114.253:33533)
0807 06:40:47.156289 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.156292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.156307 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:40:47.156326 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.156327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.156337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.156401 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.156700 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.157502 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.158276 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.158771 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.158853 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.160820 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.160828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.160828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.160835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.162933 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.163346 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.163350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.163350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.163419 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.163623 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.163625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.163625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.163831 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.163926 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.164064 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.164153 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260807 06:40:47.165091 30578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:40:47.165503 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.165350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49702)
0807 06:40:47.165416 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.165471 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 06:40:47.165700 30510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
I20260807 06:40:47.166195 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.166081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49708)
0807 06:40:47.166138 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.166171 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 06:40:47.166343 30578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:39131: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab.
Entry for principal kudu/127.11.114.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab.
I20260807 06:40:47.174319 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.252:39131
--webserver_interface=127.11.114.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab
--principal=kudu/127.11.114.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:35025,127.11.114.253:33533,127.11.114.252:39131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:40:47.213215 30469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } attempt: 3
I20260807 06:40:47.215570 30578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } attempt: 1
I20260807 06:40:47.216009 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.215853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49720)
0807 06:40:47.215917 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.215964 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 06:40:47.216812 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.216704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49722)
0807 06:40:47.216750 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.216783 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 06:40:47.216959 30578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:39131: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084846, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:40:47.221371 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.213680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54800)
0807 06:40:47.213747 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.213761 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:47.213817 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:47.214031 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:47.214032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:47.214197 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:47.214266 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.214268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.215282 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.215283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.215939 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.215942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.216436 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.216437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.216525 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.217045 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:47.217051 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:47.218337 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:47.220160 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.220161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.220161 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.220338 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.220641 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.220642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.220643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.220710 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.221049 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:47.221050 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.221143 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.221217 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.221306 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 06:40:47.221441 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.213778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54800 (local address 127.11.114.253:33533)
0807 06:40:47.213851 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.213854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.213867 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:40:47.213902 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.213904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.213921 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.214009 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.214348 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.215183 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.216027 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.216351 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.216425 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.218420 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.218421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.218422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.218429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.220040 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.220436 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.220437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.220438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.220510 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.220791 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.220792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.220793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.220937 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.221012 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.221263 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.221360 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:40:47.222078 30469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:40:47.222409 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.222315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49726)
0807 06:40:47.222359 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.222389 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 06:40:47.222780 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.222688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49732)
0807 06:40:47.222730 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.222762 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 06:40:47.222865 30469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:39131: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
I20260807 06:40:47.254040 30469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } attempt: 1
I20260807 06:40:47.254539 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.254398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49740)
0807 06:40:47.254469 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.254509 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 06:40:47.254889 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.254789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49752)
0807 06:40:47.254836 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.254868 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:40:47.255010 30469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:39131: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
W20260807 06:40:47.264204 30588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:47.264427 30588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:47.264467 30588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:47.265663 30588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:47.265731 30588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:47.265756 30588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:47.265784 30588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:47.265816 30588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:47.267346 30588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:35025,127.11.114.253:33533,127.11.114.252:39131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab
--principal=kudu/127.11.114.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.252:39131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.114.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:47.267627 30588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:47.267872 30588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:47.273070 30598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:47.273094 30594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:47.273078 30595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:47.273554 30588 server_base.cc:1034] running on GCE node
I20260807 06:40:47.273823 30588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:47.274173 30588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:47.275322 30588 hybrid_clock.cc:648] HybridClock initialized: now 1786084847275304 us; error 55 us; skew 500 ppm
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084847, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:47.278658 30588 init.cc:377] Logged in from keytab as kudu/127.11.114.252@KRBTEST.COM (short username kudu)
I20260807 06:40:47.279157 30588 webserver.cc:460] Webserver started at http://127.11.114.252:40249/ using document root <none> and password file <none>
I20260807 06:40:47.279424 30588 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:47.279471 30588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:47.279590 30588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:40:47.280937 30588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/data/instance:
uuid: "6d3e0759e84941c6ba5946160302057e"
format_stamp: "Formatted at 2026-08-07 06:40:47 on dist-test-slave-gkw7"
server_key: "85598830f6f741734dd96aab64d09bc7"
server_key_iv: "75f7bcefc7a9bf7fa43c9ede3e293d35"
server_key_version: "encryptionkey@0"
I20260807 06:40:47.281342 30588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/wal/instance:
uuid: "6d3e0759e84941c6ba5946160302057e"
format_stamp: "Formatted at 2026-08-07 06:40:47 on dist-test-slave-gkw7"
server_key: "85598830f6f741734dd96aab64d09bc7"
server_key_iv: "75f7bcefc7a9bf7fa43c9ede3e293d35"
server_key_version: "encryptionkey@0"
I20260807 06:40:47.283450 30588 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260807 06:40:47.285079 30610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:47.285113 30578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } attempt: 2
I20260807 06:40:47.285321 30588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:47.285451 30588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/wal
uuid: "6d3e0759e84941c6ba5946160302057e"
format_stamp: "Formatted at 2026-08-07 06:40:47 on dist-test-slave-gkw7"
server_key: "85598830f6f741734dd96aab64d09bc7"
server_key_iv: "75f7bcefc7a9bf7fa43c9ede3e293d35"
server_key_version: "encryptionkey@0"
I20260807 06:40:47.285507 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.285381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49764)
0807 06:40:47.285439 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.285469 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 06:40:47.285607 30588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:47.286187 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.286097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49768)
0807 06:40:47.286139 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.286161 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 06:40:47.286311 30578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:39131: Network error: Client connection negotiation failed: client connection to 127.11.114.252:39131: connect: Connection refused (error 111)
I20260807 06:40:47.320050 30588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:47.322204 30588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:47.322383 30588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:47.326725 30588 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.252:39131
I20260807 06:40:47.326769 30691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.252:39131 every 8 connection(s)
I20260807 06:40:47.327127 30588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
I20260807 06:40:47.328027 30692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:47.329682 30692 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:40:47.330197 30469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } attempt: 2
I20260807 06:40:47.333901 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 30588
I20260807 06:40:47.334080 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084684348861-11723-0/minicluster-data/master-2/wal/instance
I20260807 06:40:47.334331 11723 external_mini_cluster.cc:1428] Setting key af73a21adcdd6b5967f340814efab1ed
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084846, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084847, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:47.339761 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.330554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49778)
0807 06:40:47.330639 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.330652 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:47.330708 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:47.331452 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:47.331453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:47.331650 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:47.331688 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.331690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.332986 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.332987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.333450 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.333452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.333987 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.333988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.334052 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.334554 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:47.334565 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:47.335905 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:47.338114 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.338116 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.338116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.338470 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.338740 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.338741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.338741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.338815 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.339315 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:47.339317 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.339369 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.339518 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.339617 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 06:40:47.340183 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.330488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.11.114.254:35025)
0807 06:40:47.330559 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.330562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.330852 (+ 290us) server_negotiation.cc:407] Connection header received
0807 06:40:47.331452 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.331453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.331465 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.331526 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.332122 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.332572 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.333557 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.334587 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.334696 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.337170 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.337172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.337173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.337180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.338958 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.339353 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.339354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.339354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.339414 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.339638 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.339639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.339639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.339753 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.339829 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.340015 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.340097 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:40:47.340399 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.330505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:35025 (local address 127.0.0.1:42622)
0807 06:40:47.330700 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.330751 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:47.331294 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:47.331562 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:47.331565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:47.331912 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:47.332027 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.332032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.332656 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.332657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.333436 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.333438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.334708 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.334709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.334795 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.335408 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:47.335415 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:47.337093 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:47.339036 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.339041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.339043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.339272 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.339499 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.339500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.339500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.339565 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.339821 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:47.339828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.339901 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.339996 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.340156 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260807 06:40:47.340585 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.330630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.11.114.252:39131)
0807 06:40:47.330793 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.330797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.330828 (+ 31us) server_negotiation.cc:407] Connection header received
0807 06:40:47.331296 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.331310 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.331349 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.331442 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.331776 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.332897 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.333584 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.333898 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.334065 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.335994 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.336002 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.336002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.336009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.338019 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.338560 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.338563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.338564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.338682 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.338888 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.338889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.338889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.339227 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.339317 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.340134 (+ 817us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.340466 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:40:47.341079 30692 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:40:47.342937 30469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82: Bootstrap starting.
I20260807 06:40:47.343855 30469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:47.344218 30469 log.cc:824] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084847, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:40:47.345155 30469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82: No bootstrap required, opened a new log
I20260807 06:40:47.345335 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.334945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.11.114.252:39131)
0807 06:40:47.335211 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.335213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.335229 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:40:47.335249 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.335250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.335261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.335331 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.336567 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.337253 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.338344 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.338568 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.338645 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.341402 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.341403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.341403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.341410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.343940 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.344384 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.344385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.344386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.344450 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.344703 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.344705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.344705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.344841 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.345012 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.345087 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.345175 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
I20260807 06:40:47.347476 30469 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.347627 30469 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:47.347693 30469 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4315940690b465ba0beb541c2b85b82, State: Initialized, Role: FOLLOWER
I20260807 06:40:47.347834 30469 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.348150 30704 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.348204 30469 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:47.348630 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.341433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.11.114.253:33533)
0807 06:40:47.341480 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.341483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.341514 (+ 31us) server_negotiation.cc:407] Connection header received
0807 06:40:47.341610 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.341611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.341627 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.341673 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.341965 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.342420 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.343292 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.343504 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.343594 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.345555 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.345556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.345557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.345564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.347295 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.347702 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.347704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.347704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.347765 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.347982 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.347983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.347983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.348127 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.348183 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.348417 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.348515 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:40:47.348990 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.341338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:33533 (local address 127.0.0.1:54806)
0807 06:40:47.341472 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.341484 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:47.341547 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:47.341684 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:47.341685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:47.341828 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:47.341873 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.341875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.342503 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.342504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.343160 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.343163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.343593 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.343594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.343664 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.344049 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:47.344054 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:47.345428 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:47.347392 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.347393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.347393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.347602 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.347843 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.347844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.347845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.347904 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.348235 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:47.348236 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.348278 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.348749 (+ 471us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.348855 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260807 06:40:47.349351 30692 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
W20260807 06:40:47.350432 30723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:40:47.350493 30723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:47.348282 30704 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:40:47.351161 30368 master_runner.cc:186] Error getting master registration for 127.11.114.253:33533: OK, instance_id { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" instance_seqno: 1786084847139844 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:40:47.354038 30368 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:40:47.354513 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.345976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.11.114.254:35025)
0807 06:40:47.346013 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.346015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.346100 (+ 85us) server_negotiation.cc:407] Connection header received
0807 06:40:47.346170 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.346171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.346182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.346225 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.346850 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.347297 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.348197 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.348499 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.348534 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.351266 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.351267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.351268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.351275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.352979 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.353387 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.353388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.353389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.353453 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.353743 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.353744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.353744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.353875 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.353989 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.354336 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.354429 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084847, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:40:47.358490 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.349657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49798)
0807 06:40:47.349760 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.349769 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:47.349811 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:47.349964 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:47.349964 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:47.350097 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:47.350144 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.350146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.351097 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.351097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.352455 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.352458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.352969 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.352970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.353022 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.353441 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:47.353446 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:47.355298 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:47.357109 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.357110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.357111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.357394 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.357673 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.357674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.357674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.357750 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.358106 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:47.358107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.358161 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.358241 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.358359 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 06:40:47.358711 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.349706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.11.114.252:39131)
0807 06:40:47.349739 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.349741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.349862 (+ 121us) server_negotiation.cc:407] Connection header received
0807 06:40:47.349880 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.349880 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.349890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.349933 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.350226 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.350923 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.352544 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.352861 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.352926 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.355404 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.355406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.355406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.355413 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.356983 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.357487 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.357489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.357489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.357583 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.357828 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.357829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.357830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.358014 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.358089 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.358295 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.358419 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":16}
I20260807 06:40:47.360596 30692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e: Bootstrap starting.
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:40:47.361502 30692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:47.362167 30692 log.cc:824] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:47.363438 30692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e: No bootstrap required, opened a new log
I20260807 06:40:47.364938 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.355312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.11.114.253:33533)
0807 06:40:47.355349 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.355351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.355359 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:40:47.355410 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.355410 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.355418 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.355458 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.356201 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.357171 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.358187 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.358481 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.358542 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.361669 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.361671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.361671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.361680 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.363467 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.363984 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.363985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.363985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.364067 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.364307 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.364309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.364310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.364446 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.364669 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.364714 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.364802 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:40:47.366068 30692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.366226 30692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:47.366263 30692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d3e0759e84941c6ba5946160302057e, State: Initialized, Role: FOLLOWER
I20260807 06:40:47.366382 30692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.366707 30692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:47.366874 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:40:47.366690 30729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.368399 30729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:40:47.369036 30759 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:40:47.369149 30759 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:40:47.369814 30588 master_runner.cc:186] Error getting master registration for 127.11.114.253:33533: OK, instance_id { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" instance_seqno: 1786084847139844 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:40:47.372241 30588 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:40:47.377449 30578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } attempt: 3
I20260807 06:40:47.381095 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.370192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.11.114.252:39131)
0807 06:40:47.370236 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.370238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.372781 (+ 2543us) server_negotiation.cc:407] Connection header received
0807 06:40:47.372879 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.372880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.372890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.372919 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.373675 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.374742 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.376029 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.376582 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.376627 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.378126 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.378128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.378128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.378135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.379781 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.380217 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.380219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.380219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.380276 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.380470 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.380471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.380471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.380591 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.380693 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.380900 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.380999 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:40:47.381258 30729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:40:47.381366 30729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.381731 30729 leader_election.cc:288] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4315940690b465ba0beb541c2b85b82 (127.11.114.254:35025), 09215efa3c7a408aa85e2de4cb7f7c92 (127.11.114.253:33533)
I20260807 06:40:47.382536 30428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d3e0759e84941c6ba5946160302057e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4315940690b465ba0beb541c2b85b82" is_pre_election: true
I20260807 06:40:47.382709 30428 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d3e0759e84941c6ba5946160302057e in term 0.
I20260807 06:40:47.383140 30614 leader_election.cc:302] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d3e0759e84941c6ba5946160302057e, d4315940690b465ba0beb541c2b85b82; no voters:
I20260807 06:40:47.383302 30729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 0 FOLLOWER]: Leader pre-election won for term 1
Aug 07 06:40:47 dist-test-slave-gkw7 krb5kdc[30348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084847, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:40:47.383371 30729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:40:47.383405 30729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:47.384513 30729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.384719 30729 leader_election.cc:288] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [CANDIDATE]: Term 1 election: Requested vote from peers d4315940690b465ba0beb541c2b85b82 (127.11.114.254:35025), 09215efa3c7a408aa85e2de4cb7f7c92 (127.11.114.253:33533)
I20260807 06:40:47.385265 30428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d3e0759e84941c6ba5946160302057e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4315940690b465ba0beb541c2b85b82"
I20260807 06:40:47.385389 30428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:47.386608 30428 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d3e0759e84941c6ba5946160302057e in term 1.
I20260807 06:40:47.386889 30614 leader_election.cc:302] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d3e0759e84941c6ba5946160302057e, d4315940690b465ba0beb541c2b85b82; no voters:
I20260807 06:40:47.387024 30729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:40:47.387261 30729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [term 1 LEADER]: Becoming Leader. State: Replica: 6d3e0759e84941c6ba5946160302057e, State: Running, Role: LEADER
I20260807 06:40:47.387377 30729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.387899 30729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d3e0759e84941c6ba5946160302057e. Latest consensus state: current_term: 1 leader_uuid: "6d3e0759e84941c6ba5946160302057e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.388022 30729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:47.388917 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.369909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.11.114.254:35025)
0807 06:40:47.370078 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.370080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.370093 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:40:47.370214 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.370216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.370230 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.370271 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.371229 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.372005 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.373268 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.373606 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.373664 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.375436 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.375437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.375438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.375447 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.377306 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.387466 (+ 10160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.387467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.387468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.387610 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.387984 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.387986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.387986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.388184 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.388285 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.388697 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.388805 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260807 06:40:47.389344 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.372496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.11.114.253:33533)
0807 06:40:47.372533 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.372535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.381733 (+ 9198us) server_negotiation.cc:407] Connection header received
0807 06:40:47.381858 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.381859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.381870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.381899 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.382428 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.383139 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.384101 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.384393 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.384432 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.386298 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.386299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.386299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.386306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.387929 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.388365 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.388366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.388366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.388419 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.388610 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.388611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.388611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.388902 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.389016 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.389171 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.389252 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:40:47.389798 30770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:40:47.390019 30770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:40:47.390223 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.377800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:39131 (local address 127.0.0.1:49810)
0807 06:40:47.377928 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:47.377940 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:47.377986 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:47.378321 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:47.378321 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:47.378508 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:47.378559 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.378561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.379539 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.379540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.380349 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:47.380352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.381069 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.381070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:47.381151 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.381751 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:47.381758 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:47.384094 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:47.386460 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.386461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.386462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.386726 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.388560 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:47.388561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:47.388561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:47.388672 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:47.389844 (+ 1172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:47.389845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:47.389916 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 06:40:47.389981 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:40:47.390111 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 06:40:47.391130 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:47.378107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.11.114.252:39131)
0807 06:40:47.378155 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:47.378157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:47.378165 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:40:47.378188 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:47.378189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:47.378211 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:47.378272 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:47.378660 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.379416 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.380469 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:47.380874 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:47.380973 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:47.384212 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:47.384214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:47.384214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:47.384222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:47.386341 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.386885 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.386886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.386887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.388376 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:47.389166 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:47.389167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:47.389167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:47.389661 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:47.390503 (+ 842us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:47.390645 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 06:40:47.390851 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:40:47.392611 30578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92: Bootstrap starting.
I20260807 06:40:47.393420 30578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:47.393934 30578 log.cc:824] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:47.394884 30578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92: No bootstrap required, opened a new log
I20260807 06:40:47.397306 30578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.397482 30578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:47.397588 30578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09215efa3c7a408aa85e2de4cb7f7c92, State: Initialized, Role: FOLLOWER
I20260807 06:40:47.397789 30578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } }
I20260807 06:40:47.398181 30578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:47.398221 30541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d3e0759e84941c6ba5946160302057e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09215efa3c7a408aa85e2de4cb7f7c92"
I20260807 06:40:47.398320 30541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:47.398998 30428 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [term 1 FOLLOWER]: Refusing update from remote peer 6d3e0759e84941c6ba5946160302057e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:40:47.399283 30768 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:40:47.399426 30541 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d3e0759e84941c6ba5946160302057e in term 1.
I20260807 06:40:47.399597 30543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d3e0759e84941c6ba5946160302057e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" is_pre_election: true
I20260807 06:40:47.399690 30543 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6d3e0759e84941c6ba5946160302057e in term 1. Re-sending same reply.
I20260807 06:40:47.400379 30540 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [term 1 FOLLOWER]: Refusing update from remote peer 6d3e0759e84941c6ba5946160302057e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:40:47.400517 30704 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d3e0759e84941c6ba5946160302057e. Latest consensus state: current_term: 1 leader_uuid: "6d3e0759e84941c6ba5946160302057e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.400595 30704 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:40:47.401024 30790 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:40:47.401090 30790 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:40:47.401252 30768 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:40:47.401618 30776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.401834 30776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:40:47.403930 30768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d3e0759e84941c6ba5946160302057e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.404012 30768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:47.404522 30774 mvcc.cc:205] Tried to move back new op lower bound from 7315803534941704192 to 7315803534899687424. Current Snapshot: MvccSnapshot[applied={T|T < 7315803534941704192}]
I20260807 06:40:47.404621 30776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d3e0759e84941c6ba5946160302057e. Latest consensus state: current_term: 1 leader_uuid: "6d3e0759e84941c6ba5946160302057e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.404692 30776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:40:47.404971 30704 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d3e0759e84941c6ba5946160302057e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.405077 30704 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:40:47.406251 30793 mvcc.cc:205] Tried to move back new op lower bound from 7315803534941704192 to 7315803534899687424. Current Snapshot: MvccSnapshot[applied={T|T < 7315803534941704192}]
I20260807 06:40:47.406952 30770 catalog_manager.cc:1347] Generated new cluster ID: 73f7bdc6262149ca9264bf892bfe8451
I20260807 06:40:47.407011 30770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:40:47.411634 30729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d3e0759e84941c6ba5946160302057e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.411737 30729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:47.412429 30776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d3e0759e84941c6ba5946160302057e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4315940690b465ba0beb541c2b85b82" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 35025 } } peers { permanent_uuid: "09215efa3c7a408aa85e2de4cb7f7c92" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 33533 } } peers { permanent_uuid: "6d3e0759e84941c6ba5946160302057e" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 39131 } } }
I20260807 06:40:47.412513 30776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:40:47.419199 30770 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:40:47.419940 30770 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:40:47.429621 30770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d3e0759e84941c6ba5946160302057e: Generated new TSK 0
I20260807 06:40:47.429919 30770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:40:48.350939 30723 catalog_manager.cc:1259] Loaded cluster ID: 73f7bdc6262149ca9264bf892bfe8451
I20260807 06:40:48.351016 30723 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82: loading cluster ID for follower catalog manager: success
I20260807 06:40:48.351840 30723 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82: acquiring CA information for follower catalog manager: success
I20260807 06:40:48.352115 30723 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4315940690b465ba0beb541c2b85b82: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:40:48.401659 30790 catalog_manager.cc:1259] Loaded cluster ID: 73f7bdc6262149ca9264bf892bfe8451
I20260807 06:40:48.401768 30790 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92: loading cluster ID for follower catalog manager: success
I20260807 06:40:48.403085 30790 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92: acquiring CA information for follower catalog manager: success
I20260807 06:40:48.403414 30790 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09215efa3c7a408aa85e2de4cb7f7c92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:40:48.491190 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 30368
I20260807 06:40:48.503530 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 30477
I20260807 06:40:48.512912 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 30588
2026-08-07T06:40:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3003 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:40:48 dist-test-slave-gkw7 krb5kdc[30815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:40:48 dist-test-slave-gkw7 krb5kdc[30815](info): set up 2 sockets
Aug 07 06:40:48 dist-test-slave-gkw7 krb5kdc[30815](info): commencing operation
krb5kdc: starting...
W20260807 06:40:49.815791 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:40:49 dist-test-slave-gkw7 krb5kdc[30815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:40:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:40:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:40:49.950404 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:44971
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:44971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:50.034291 30831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:50.034503 30831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:50.034528 30831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:50.035705 30831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:50.035748 30831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:50.035763 30831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:50.035775 30831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:50.035786 30831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:50.037328 30831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:44971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:44971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:50.037577 30831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:50.037792 30831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:50.042840 30838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:50.042846 30843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:50.042905 30840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:50.043025 30831 server_base.cc:1034] running on GCE node
I20260807 06:40:50.043363 30831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:50.043674 30831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:50.044866 30831 hybrid_clock.cc:648] HybridClock initialized: now 1786084850044856 us; error 36 us; skew 500 ppm
Aug 07 06:40:50 dist-test-slave-gkw7 krb5kdc[30815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084850, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:50.047559 30831 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:40:50.048017 30831 webserver.cc:460] Webserver started at http://127.11.114.254:41501/ using document root <none> and password file <none>
I20260807 06:40:50.048281 30831 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:50.048353 30831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:50.048506 30831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:40:50.049654 30831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "30947cc62dc443f2b5e801b046c4a4de"
format_stamp: "Formatted at 2026-08-07 06:40:50 on dist-test-slave-gkw7"
server_key: "b2283c3f0b46f348583bf45901aa17aa"
server_key_iv: "ae3beb0517aa4f13d6e7801e135ee4aa"
server_key_version: "encryptionkey@0"
I20260807 06:40:50.050025 30831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "30947cc62dc443f2b5e801b046c4a4de"
format_stamp: "Formatted at 2026-08-07 06:40:50 on dist-test-slave-gkw7"
server_key: "b2283c3f0b46f348583bf45901aa17aa"
server_key_iv: "ae3beb0517aa4f13d6e7801e135ee4aa"
server_key_version: "encryptionkey@0"
I20260807 06:40:50.051461 30831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:40:50.052292 30855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:50.052510 30831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:40:50.052624 30831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "30947cc62dc443f2b5e801b046c4a4de"
format_stamp: "Formatted at 2026-08-07 06:40:50 on dist-test-slave-gkw7"
server_key: "b2283c3f0b46f348583bf45901aa17aa"
server_key_iv: "ae3beb0517aa4f13d6e7801e135ee4aa"
server_key_version: "encryptionkey@0"
I20260807 06:40:50.052716 30831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:50.070360 30831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:50.072368 30831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:50.072547 30831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:50.076642 30831 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:44971
I20260807 06:40:50.076679 30937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:44971 every 8 connection(s)
I20260807 06:40:50.077068 30831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:40:50.078085 30938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:50.080440 30938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de: Bootstrap starting.
I20260807 06:40:50.080992 30938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:50.081235 30938 log.cc:824] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:50.082021 30938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de: No bootstrap required, opened a new log
I20260807 06:40:50.083595 30938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30947cc62dc443f2b5e801b046c4a4de" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44971 } }
I20260807 06:40:50.083740 30938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:50.083763 30938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30947cc62dc443f2b5e801b046c4a4de, State: Initialized, Role: FOLLOWER
I20260807 06:40:50.083914 30938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30947cc62dc443f2b5e801b046c4a4de" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44971 } }
I20260807 06:40:50.083999 30938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:40:50.084023 30938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:40:50.084053 30938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:50.084575 30938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30947cc62dc443f2b5e801b046c4a4de" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44971 } }
I20260807 06:40:50.084662 30938 leader_election.cc:302] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30947cc62dc443f2b5e801b046c4a4de; no voters:
I20260807 06:40:50.084810 30938 leader_election.cc:288] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:40:50.084883 30941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:40:50.085074 30938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:50.085095 30941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [term 1 LEADER]: Becoming Leader. State: Replica: 30947cc62dc443f2b5e801b046c4a4de, State: Running, Role: LEADER
I20260807 06:40:50.085281 30941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30947cc62dc443f2b5e801b046c4a4de" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44971 } }
I20260807 06:40:50.085907 30943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30947cc62dc443f2b5e801b046c4a4de. Latest consensus state: current_term: 1 leader_uuid: "30947cc62dc443f2b5e801b046c4a4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30947cc62dc443f2b5e801b046c4a4de" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44971 } } }
I20260807 06:40:50.086017 30943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:50.086167 30942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30947cc62dc443f2b5e801b046c4a4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30947cc62dc443f2b5e801b046c4a4de" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44971 } } }
I20260807 06:40:50.086244 30942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:50.086333 30954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:40:50.086376 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 30831
I20260807 06:40:50.086478 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:40:50.086690 11723 external_mini_cluster.cc:1428] Setting key 98021615216cd9627211de732b803d80
I20260807 06:40:50.086894 30954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:40:50.088747 30954 catalog_manager.cc:1347] Generated new cluster ID: fb0f4a7b335d4d15b7f695e46bb21fd7
I20260807 06:40:50.088794 30954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:40:50 dist-test-slave-gkw7 krb5kdc[30815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:50.099128 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:50.087339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.11.114.254:44971)
0807 06:40:50.087521 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:50.087531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:50.087563 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:40:50.087668 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:50.087673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:50.087749 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:50.087891 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:50.088879 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:50.090320 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:50.091455 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:50.092098 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:50.092207 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:50.095285 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:50.095290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:50.095291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:50.095306 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:50.097362 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:50.097783 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:50.097786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:50.097787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:50.097918 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:50.098129 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:50.098130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:50.098131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:50.098465 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:50.098622 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:50.098723 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 06:40:50.098875 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260807 06:40:50.100741 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:40:50.102723 30954 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:40:50.103258 30954 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:40:50.113996 30954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30947cc62dc443f2b5e801b046c4a4de: Generated new TSK 0
I20260807 06:40:50.114238 30954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:40:50.155515 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 30831
2026-08-07T06:40:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1645 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:40:50 dist-test-slave-gkw7 krb5kdc[30976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:40:50 dist-test-slave-gkw7 krb5kdc[30976](info): set up 2 sockets
Aug 07 06:40:50 dist-test-slave-gkw7 krb5kdc[30976](info): commencing operation
krb5kdc: starting...
W20260807 06:40:51.378494 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:40:51 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:40:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:40:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:40:51.504200 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40759
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:40759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:51.614203 30997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:51.614439 30997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:51.614478 30997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:51.615748 30997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:51.615823 30997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:51.615847 30997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:51.615873 30997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:51.615904 30997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:51.617395 30997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:40759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:51.617689 30997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:51.617931 30997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:40:51.622967 30997 server_base.cc:1034] running on GCE node
W20260807 06:40:51.622920 31008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:51.622924 31010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:51.622946 31007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:51.623469 30997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:51.623777 30997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:51.624967 30997 hybrid_clock.cc:648] HybridClock initialized: now 1786084851624952 us; error 48 us; skew 500 ppm
Aug 07 06:40:51 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084851, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:51.627785 30997 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:40:51.628232 30997 webserver.cc:460] Webserver started at http://127.11.114.254:35145/ using document root <none> and password file <none>
I20260807 06:40:51.628491 30997 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:51.628561 30997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:51.628723 30997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:40:51.629871 30997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "ce48457dc21f459db3d90883a78ae502"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "8cbaba356e47c4753e063b25c9226b61"
server_key_iv: "a41056a76eed4ee9671c12ffc068cce8"
server_key_version: "encryptionkey@0"
I20260807 06:40:51.630236 30997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "ce48457dc21f459db3d90883a78ae502"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "8cbaba356e47c4753e063b25c9226b61"
server_key_iv: "a41056a76eed4ee9671c12ffc068cce8"
server_key_version: "encryptionkey@0"
I20260807 06:40:51.631692 30997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:51.632608 31025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:51.632800 30997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:51.632912 30997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "ce48457dc21f459db3d90883a78ae502"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "8cbaba356e47c4753e063b25c9226b61"
server_key_iv: "a41056a76eed4ee9671c12ffc068cce8"
server_key_version: "encryptionkey@0"
I20260807 06:40:51.632988 30997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:51.656605 30997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:51.659255 30997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:51.659442 30997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:51.663558 30997 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:40759
I20260807 06:40:51.663618 31103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:40759 every 8 connection(s)
I20260807 06:40:51.664027 30997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:40:51.665068 31104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:51.667423 31104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Bootstrap starting.
I20260807 06:40:51.668030 31104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:51.668329 31104 log.cc:824] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:51.669117 31104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: No bootstrap required, opened a new log
I20260807 06:40:51.670738 31104 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:51.670881 31104 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:51.670903 31104 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce48457dc21f459db3d90883a78ae502, State: Initialized, Role: FOLLOWER
I20260807 06:40:51.671067 31104 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:51.671132 31104 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:40:51.671154 31104 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:40:51.671182 31104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:51.671630 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 30997
I20260807 06:40:51.671739 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:40:51.671985 11723 external_mini_cluster.cc:1428] Setting key a690901f446dee5f142c110fe308414b
I20260807 06:40:51.672235 31104 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:51.672362 31104 leader_election.cc:302] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce48457dc21f459db3d90883a78ae502; no voters:
I20260807 06:40:51.672549 31104 leader_election.cc:288] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:40:51.672627 31111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:40:51.672778 31111 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 1 LEADER]: Becoming Leader. State: Replica: ce48457dc21f459db3d90883a78ae502, State: Running, Role: LEADER
I20260807 06:40:51.672847 31104 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:51.672868 31111 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:51.673197 31116 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce48457dc21f459db3d90883a78ae502. Latest consensus state: current_term: 1 leader_uuid: "ce48457dc21f459db3d90883a78ae502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } } }
I20260807 06:40:51.673311 31116 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:51.673187 31117 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce48457dc21f459db3d90883a78ae502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } } }
I20260807 06:40:51.673507 31117 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:51.674290 31125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:40:51.674906 31125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:40:51.676478 31125 catalog_manager.cc:1347] Generated new cluster ID: 7def937d07e640b7b37bc26a2f19a286
I20260807 06:40:51.676589 31125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:40:51 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:51.684947 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:51.672564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.11.114.254:40759)
0807 06:40:51.672794 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:51.672805 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:51.673343 (+ 538us) server_negotiation.cc:407] Connection header received
0807 06:40:51.673610 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:51.673618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:51.673675 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:51.673774 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:51.674874 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:51.676399 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:51.677755 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:51.678807 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:51.678941 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:51.681383 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:51.681387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:51.681387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:51.681401 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:51.683307 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:51.683693 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:51.683696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:51.683697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:51.683817 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:51.684036 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:51.684037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:51.684038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:51.684300 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:51.684483 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:51.684555 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 06:40:51.684716 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":102,"threads_started":1}
I20260807 06:40:51.692039 31125 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:40:51.692756 31125 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:40:51.704653 31125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Generated new TSK 0
I20260807 06:40:51.704898 31125 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:40:51.732287 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40759
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:51.836228 31148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:51.836516 31148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:51.836567 31148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:51.837935 31148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:51.838002 31148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:51.838048 31148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:40:51.839543 31148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:51.839829 31148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:51.840075 31148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:51.845224 31154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:51.845225 31155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:51.845216 31160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:51.845672 31148 server_base.cc:1034] running on GCE node
I20260807 06:40:51.845947 31148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:51.846199 31148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:51.847364 31148 hybrid_clock.cc:648] HybridClock initialized: now 1786084851847338 us; error 39 us; skew 500 ppm
Aug 07 06:40:51 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084851, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:51.850103 31148 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:40:51.850555 31148 webserver.cc:460] Webserver started at http://127.11.114.193:45687/ using document root <none> and password file <none>
I20260807 06:40:51.850806 31148 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:51.850876 31148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:51.851020 31148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:40:51.852073 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "7a7c2eec402eb13e65ce77daeba8c6e6"
server_key_iv: "d2acdb813c4c719b02ab2e1dc10991ee"
server_key_version: "encryptionkey@0"
I20260807 06:40:51.852440 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "7a7c2eec402eb13e65ce77daeba8c6e6"
server_key_iv: "d2acdb813c4c719b02ab2e1dc10991ee"
server_key_version: "encryptionkey@0"
I20260807 06:40:51.853914 31148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:40:51.854705 31167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:51.854902 31148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:51.855005 31148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "7a7c2eec402eb13e65ce77daeba8c6e6"
server_key_iv: "d2acdb813c4c719b02ab2e1dc10991ee"
server_key_version: "encryptionkey@0"
I20260807 06:40:51.855078 31148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:51.875340 31148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:51.877549 31148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:51.877730 31148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:51.877982 31148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:51.878314 31148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:51.878381 31148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:51.878422 31148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:51.878449 31148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:51.884449 31148 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:44243
I20260807 06:40:51.884527 31340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:44243 every 8 connection(s)
I20260807 06:40:51.884886 31148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:40:51.888793 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 31148
I20260807 06:40:51.888911 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:40:51.889117 11723 external_mini_cluster.cc:1428] Setting key 505604c66a049b144fe45df0c182eccc
Aug 07 06:40:51 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084851, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:51.896050 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:51.885732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:59399)
0807 06:40:51.885911 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:51.885955 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:51.886459 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:51.886721 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:51.886724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:51.887326 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:51.887526 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:51.887530 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:51.888235 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:51.888236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:51.889411 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:51.889415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:51.889963 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:51.889964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:51.890084 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:51.890650 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:51.890658 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:51.892307 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:51.894480 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:51.894486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:51.894488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:51.894749 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:51.895023 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:51.895024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:51.895024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:51.895141 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:51.895500 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:51.895506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:51.895632 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 06:40:51.895732 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:40:51.895877 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260807 06:40:51.896040 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:51.885788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:59399 (local address 127.11.114.254:40759)
0807 06:40:51.885856 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:51.885858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:51.886060 (+ 202us) server_negotiation.cc:407] Connection header received
0807 06:40:51.886556 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:51.886558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:51.886572 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:51.886636 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:51.887630 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:51.888136 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:51.889517 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:51.889760 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:51.889854 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:51.892414 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:51.892416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:51.892416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:51.892426 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:51.894382 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:51.894834 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:51.894836 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:51.894836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:51.894905 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:51.895245 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:51.895246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:51.895247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:51.895420 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:51.895501 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:51.895798 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 06:40:51.895901 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:40:51.896735 31341 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40759
I20260807 06:40:51.896857 31341 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:51.897042 31341 heartbeater.cc:511] Master 127.11.114.254:40759 requested a full tablet report, sending...
I20260807 06:40:51.897703 31051 ts_manager.cc:194] Registered new tserver with Master: 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243)
I20260807 06:40:51.898331 31051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:59399
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:40:51.921015 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40759
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:52.005610 31346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:52.005882 31346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:52.005923 31346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:52.007110 31346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:52.007192 31346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:52.007237 31346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:40:52.008795 31346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:52.009083 31346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:52.009342 31346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:52.014745 31355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.014757 31352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.014756 31351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:52.014988 31346 server_base.cc:1034] running on GCE node
I20260807 06:40:52.015213 31346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:52.015439 31346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:52.016587 31346 hybrid_clock.cc:648] HybridClock initialized: now 1786084852016554 us; error 51 us; skew 500 ppm
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:52.023008 31346 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:40:52.023552 31346 webserver.cc:460] Webserver started at http://127.11.114.194:37261/ using document root <none> and password file <none>
I20260807 06:40:52.023806 31346 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:52.023880 31346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:52.024024 31346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:40:52.025128 31346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "56c29a8a97a84ab28ba0e8644fbe7562"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "6ad523b8aa1b87d406ed96b47724fddb"
server_key_iv: "04fca9a5772ba1be5580230d902b9f63"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.025573 31346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "56c29a8a97a84ab28ba0e8644fbe7562"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "6ad523b8aa1b87d406ed96b47724fddb"
server_key_iv: "04fca9a5772ba1be5580230d902b9f63"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.026976 31346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:52.027822 31365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.028146 31346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:52.028278 31346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "56c29a8a97a84ab28ba0e8644fbe7562"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "6ad523b8aa1b87d406ed96b47724fddb"
server_key_iv: "04fca9a5772ba1be5580230d902b9f63"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.028359 31346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:52.041196 31346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:52.043704 31346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:52.043882 31346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:52.044137 31346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:52.044481 31346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:52.044539 31346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.044584 31346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:52.044607 31346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.051254 31346 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:43839
I20260807 06:40:52.051301 31526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:43839 every 8 connection(s)
I20260807 06:40:52.051680 31346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:40:52.057189 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 31346
I20260807 06:40:52.057331 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:40:52.057572 11723 external_mini_cluster.cc:1428] Setting key 40ff09928031adfe2cc7bc9e5d0ed7f1
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:52.061921 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.052453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:39011 (local address 127.11.114.254:40759)
0807 06:40:52.052527 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.052529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.052728 (+ 199us) server_negotiation.cc:407] Connection header received
0807 06:40:52.053208 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.053209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.053221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.053276 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:52.054339 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.054840 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.055797 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.055991 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.056043 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.058568 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:52.058570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:52.058570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:52.058579 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:52.060358 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.060886 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.060887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.060888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.060945 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.061237 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.061237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.061238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.061343 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:52.061395 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.061765 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.061857 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:40:52.062230 31530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.052439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:39011)
0807 06:40:52.052598 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.052635 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.053116 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.053349 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.053352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.054052 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:52.054258 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.054263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.054922 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.054923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.055720 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.055722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.056062 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.056063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.056165 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.056657 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:52.056666 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:52.058482 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:52.060462 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.060469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.060472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.060800 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.061012 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.061013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.061014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.061166 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.061410 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:52.061417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:52.061623 (+ 206us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.061748 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.061945 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260807 06:40:52.062702 31528 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40759
I20260807 06:40:52.062800 31528 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:52.063005 31528 heartbeater.cc:511] Master 127.11.114.254:40759 requested a full tablet report, sending...
I20260807 06:40:52.063431 31051 ts_manager.cc:194] Registered new tserver with Master: 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839)
I20260807 06:40:52.063831 31051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:39011
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:40:52.086514 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40759
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:52.174407 31533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:52.174618 31533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:52.174641 31533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:52.175763 31533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:52.175806 31533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:52.175833 31533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:40:52.177315 31533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:52.177624 31533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:52.177860 31533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:52.183012 31544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.182993 31546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.182986 31542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:52.183187 31533 server_base.cc:1034] running on GCE node
I20260807 06:40:52.183449 31533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:52.183697 31533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:52.184880 31533 hybrid_clock.cc:648] HybridClock initialized: now 1786084852184862 us; error 34 us; skew 500 ppm
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:52.187510 31533 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:40:52.187988 31533 webserver.cc:460] Webserver started at http://127.11.114.195:38357/ using document root <none> and password file <none>
I20260807 06:40:52.188259 31533 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:52.188338 31533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:52.188498 31533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:40:52.189669 31533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "91a17cc31e224e88b898e8643459f3a0"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "e1246299a6ce5475df3c3ad76dd09927"
server_key_iv: "b7e72952ad7d468ae17a12b41dbf4d34"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.190037 31533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "91a17cc31e224e88b898e8643459f3a0"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "e1246299a6ce5475df3c3ad76dd09927"
server_key_iv: "b7e72952ad7d468ae17a12b41dbf4d34"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.191406 31533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:40:52.192175 31558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.192355 31533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:52.192467 31533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "91a17cc31e224e88b898e8643459f3a0"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "e1246299a6ce5475df3c3ad76dd09927"
server_key_iv: "b7e72952ad7d468ae17a12b41dbf4d34"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.192541 31533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:52.209357 31533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:52.211901 31533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:52.212077 31533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:52.212330 31533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:52.212654 31533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:52.212713 31533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.212755 31533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:52.212781 31533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.219316 31533 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:32937
I20260807 06:40:52.219373 31729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:32937 every 8 connection(s)
I20260807 06:40:52.219728 31533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:40:52.222654 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 31533
I20260807 06:40:52.222754 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:40:52.222911 11723 external_mini_cluster.cc:1428] Setting key cb0e48b38ce47e5ff51610fd47fab30d
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:52.229228 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.220500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:43947 (local address 127.11.114.254:40759)
0807 06:40:52.220585 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.220588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.220787 (+ 199us) server_negotiation.cc:407] Connection header received
0807 06:40:52.221358 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.221360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.221375 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.221428 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:52.222464 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.223230 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.224410 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.224599 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.224657 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.226802 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:52.226803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:52.226803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:52.226810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:52.227968 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.228337 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.228337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.228338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.228390 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.228606 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.228606 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.228606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.228707 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:52.228761 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.229093 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.229162 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:40:52.229480 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.220465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:43947)
0807 06:40:52.220644 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.220702 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.221242 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.221497 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.221501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.222150 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:52.222346 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.222351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.223334 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.223335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.224334 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.224338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.224676 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.224677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.224780 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.225319 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:52.225337 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:52.226719 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:52.228054 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.228060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.228061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.228271 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.228450 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.228450 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.228451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.228549 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.228796 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:52.228801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:52.228927 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.229067 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.229213 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260807 06:40:52.230031 31730 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40759
I20260807 06:40:52.230136 31730 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:52.230309 31730 heartbeater.cc:511] Master 127.11.114.254:40759 requested a full tablet report, sending...
I20260807 06:40:52.230641 31051 ts_manager.cc:194] Registered new tserver with Master: 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937)
I20260807 06:40:52.231015 31051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:43947
I20260807 06:40:52.234884 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:40:52.242318 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.236308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39742 (local address 127.11.114.254:40759)
0807 06:40:52.236369 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.236371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.236497 (+ 126us) server_negotiation.cc:407] Connection header received
0807 06:40:52.236536 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.236537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.236546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.236601 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:52.237152 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.237695 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.238399 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.238580 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.238636 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.239758 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:52.239759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:52.239759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:52.239765 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:52.241184 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.241504 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.241505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.241505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.241610 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.241807 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.241808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.241808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.241999 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:52.242087 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.242190 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.242254 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:40:52.243858 31051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:40:52.245467 31051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:40:52.251921 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.248031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43839 (local address 127.0.0.1:36702)
0807 06:40:52.248686 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.248695 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.248752 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.248946 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.248947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.248949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:52.248995 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.248997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.250012 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.250013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.250934 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.250936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.251618 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.251619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.251702 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.251716 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.251791 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.251843 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":1,"thread_start_us":75,"threads_started":1}
I20260807 06:40:52.251932 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.248209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.11.114.194:43839)
0807 06:40:52.248399 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.248402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.248798 (+ 396us) server_negotiation.cc:407] Connection header received
0807 06:40:52.248848 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.248850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.248862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.248950 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:52.249054 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.249919 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.251021 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.251526 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.251621 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.251691 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.251806 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.251846 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260807 06:40:52.252157 31752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.247796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:32937 (local address 127.0.0.1:58474)
0807 06:40:52.247981 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.248029 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.248102 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.248619 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.248622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.248624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:52.248720 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.248722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.249711 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.249712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.250985 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.250988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.251912 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.251913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.251971 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.251975 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.252006 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.252056 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":51,"threads_started":1}
I20260807 06:40:52.252491 31753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.247928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.11.114.195:32937)
0807 06:40:52.248426 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.248432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.248443 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:40:52.248487 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.248489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.248506 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.248591 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:52.248811 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.249596 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.251097 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.251812 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.252273 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.252334 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.252360 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.252397 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":93,"threads_started":1}
I20260807 06:40:52.252764 31439 tablet_service.cc:1467] Processing CreateTablet for tablet 0895d8066a0c40a49dc3f206185802ca (DEFAULT_TABLE table=test-table [id=9926611257b0405bb47ae989719283b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:52.253206 31439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0895d8066a0c40a49dc3f206185802ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:52.253235 31632 tablet_service.cc:1467] Processing CreateTablet for tablet 0895d8066a0c40a49dc3f206185802ca (DEFAULT_TABLE table=test-table [id=9926611257b0405bb47ae989719283b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:52.253649 31632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0895d8066a0c40a49dc3f206185802ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:52.253697 31750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.247800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44243 (local address 127.0.0.1:53802)
0807 06:40:52.248326 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.248338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.248406 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.249680 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.249681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.249683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:52.249724 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.249726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.251084 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.251085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.252382 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.252385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.253424 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.253425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.253488 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.253492 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.253542 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.253608 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":105,"threads_started":1}
I20260807 06:40:52.253913 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.248134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.11.114.193:44243)
0807 06:40:52.248324 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.248331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.248392 (+ 61us) server_negotiation.cc:407] Connection header received
0807 06:40:52.248509 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.248512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.248528 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.249668 (+ 1140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:52.249802 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.250949 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.252468 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.253265 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.253337 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.253385 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.253789 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.253836 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260807 06:40:52.256383 31759 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: Bootstrap starting.
I20260807 06:40:52.256553 31758 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Bootstrap starting.
I20260807 06:40:52.256929 31251 tablet_service.cc:1467] Processing CreateTablet for tablet 0895d8066a0c40a49dc3f206185802ca (DEFAULT_TABLE table=test-table [id=9926611257b0405bb47ae989719283b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:52.257246 31759 tablet_bootstrap.cc:654] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:52.257246 31758 tablet_bootstrap.cc:654] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:52.257272 31251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0895d8066a0c40a49dc3f206185802ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:52.257901 31758 log.cc:824] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:52.257901 31759 log.cc:824] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:52.258993 31758 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: No bootstrap required, opened a new log
I20260807 06:40:52.259003 31759 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: No bootstrap required, opened a new log
I20260807 06:40:52.259073 31758 ts_tablet_manager.cc:1397] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:40:52.259073 31759 ts_tablet_manager.cc:1397] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:40:52.260375 31763 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Bootstrap starting.
I20260807 06:40:52.260922 31763 tablet_bootstrap.cc:654] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:52.261112 31758 raft_consensus.cc:348] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.261307 31758 raft_consensus.cc:374] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:52.261359 31758 raft_consensus.cc:729] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c29a8a97a84ab28ba0e8644fbe7562, State: Initialized, Role: FOLLOWER
I20260807 06:40:52.261340 31759 raft_consensus.cc:348] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.261492 31759 raft_consensus.cc:374] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:52.261554 31759 raft_consensus.cc:729] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a17cc31e224e88b898e8643459f3a0, State: Initialized, Role: FOLLOWER
I20260807 06:40:52.261564 31763 log.cc:824] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:52.261520 31758 consensus_queue.cc:260] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.261714 31759 consensus_queue.cc:260] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.261873 31758 ts_tablet_manager.cc:1428] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 06:40:52.261911 31528 heartbeater.cc:503] Master 127.11.114.254:40759 was elected leader, sending a full tablet report...
I20260807 06:40:52.262068 31759 ts_tablet_manager.cc:1428] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 06:40:52.262102 31730 heartbeater.cc:503] Master 127.11.114.254:40759 was elected leader, sending a full tablet report...
I20260807 06:40:52.262575 31763 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: No bootstrap required, opened a new log
I20260807 06:40:52.262650 31763 ts_tablet_manager.cc:1397] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:40:52.264307 31763 raft_consensus.cc:348] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.264454 31763 raft_consensus.cc:374] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:52.264499 31763 raft_consensus.cc:729] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de0a5f5674d4203bf4b5aeb837dcfc1, State: Initialized, Role: FOLLOWER
I20260807 06:40:52.264640 31763 consensus_queue.cc:260] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.264911 31763 ts_tablet_manager.cc:1428] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:40:52.264986 31341 heartbeater.cc:503] Master 127.11.114.254:40759 was elected leader, sending a full tablet report...
W20260807 06:40:52.302341 31529 tablet.cc:2369] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:40:52.316833 31765 raft_consensus.cc:482] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:40:52.316968 31765 raft_consensus.cc:504] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.317322 31765 leader_election.cc:288] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937), 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243)
I20260807 06:40:52.320528 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.317724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:47995 (local address 127.11.114.193:44243)
0807 06:40:52.317775 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.317778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.317877 (+ 99us) server_negotiation.cc:407] Connection header received
0807 06:40:52.317977 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.317978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.317988 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.318053 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:52.318237 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.318702 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.319710 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.320162 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.320248 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.320259 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.320422 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.320463 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:40:52.320539 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.317556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44243 (local address 127.11.114.194:47995)
0807 06:40:52.317791 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.317801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.317904 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.318070 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.318070 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.318072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:52.318158 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.318159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.318804 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.318805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.319597 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.319599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.320247 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.320248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.320327 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.320331 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.320403 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.320445 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":26,"thread_start_us":130,"threads_started":1}
I20260807 06:40:52.320835 31530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.317556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:32937 (local address 127.11.114.194:35239)
0807 06:40:52.317618 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.317632 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.317700 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.317880 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.317881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.317883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:52.317966 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.317967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.318662 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.318663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.319956 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.319959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.320602 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.320603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.320687 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.320691 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.320740 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.320788 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 06:40:52.320865 31276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0895d8066a0c40a49dc3f206185802ca" candidate_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" is_pre_election: true
I20260807 06:40:52.321079 31276 raft_consensus.cc:2454] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56c29a8a97a84ab28ba0e8644fbe7562 in term 0.
I20260807 06:40:52.321272 31753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.317695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:35239 (local address 127.11.114.195:32937)
0807 06:40:52.317745 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.317747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.317755 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:40:52.317784 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.317784 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.317794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.317875 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:52.318042 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.318567 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.320051 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.320482 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.320557 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.320565 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.321135 (+ 570us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.321181 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:40:52.321372 31370 leader_election.cc:302] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de0a5f5674d4203bf4b5aeb837dcfc1, 56c29a8a97a84ab28ba0e8644fbe7562; no voters:
I20260807 06:40:52.321504 31765 raft_consensus.cc:2781] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:40:52.321588 31765 raft_consensus.cc:482] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:40:52.321621 31765 raft_consensus.cc:3037] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:52.321612 31660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0895d8066a0c40a49dc3f206185802ca" candidate_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a17cc31e224e88b898e8643459f3a0" is_pre_election: true
I20260807 06:40:52.321784 31660 raft_consensus.cc:2454] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56c29a8a97a84ab28ba0e8644fbe7562 in term 0.
I20260807 06:40:52.322489 31765 raft_consensus.cc:504] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.322644 31765 leader_election.cc:288] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [CANDIDATE]: Term 1 election: Requested vote from peers 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937), 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243)
I20260807 06:40:52.322839 31660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0895d8066a0c40a49dc3f206185802ca" candidate_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a17cc31e224e88b898e8643459f3a0"
I20260807 06:40:52.322831 31276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0895d8066a0c40a49dc3f206185802ca" candidate_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1"
I20260807 06:40:52.322906 31276 raft_consensus.cc:3037] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:52.322928 31660 raft_consensus.cc:3037] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:52.323632 31660 raft_consensus.cc:2454] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56c29a8a97a84ab28ba0e8644fbe7562 in term 1.
I20260807 06:40:52.323637 31276 raft_consensus.cc:2454] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56c29a8a97a84ab28ba0e8644fbe7562 in term 1.
I20260807 06:40:52.323838 31370 leader_election.cc:302] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de0a5f5674d4203bf4b5aeb837dcfc1, 56c29a8a97a84ab28ba0e8644fbe7562; no voters:
I20260807 06:40:52.323952 31765 raft_consensus.cc:2781] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:40:52.324110 31765 raft_consensus.cc:686] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 LEADER]: Becoming Leader. State: Replica: 56c29a8a97a84ab28ba0e8644fbe7562, State: Running, Role: LEADER
I20260807 06:40:52.324209 31765 consensus_queue.cc:237] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.325133 31051 catalog_manager.cc:5495] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } health_report { overall_health: HEALTHY } } }
I20260807 06:40:52.345259 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.342536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.11.114.194:43839)
0807 06:40:52.342618 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.342621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.342632 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:40:52.342709 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.342710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.342733 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.342836 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:40:52.342998 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.343497 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.344316 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.344548 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.344640 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.344820 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:40:52.344944 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:40:52.345043 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.345116 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.345138 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:40:52.347292 31660 raft_consensus.cc:1261] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 1 FOLLOWER]: Refusing update from remote peer 56c29a8a97a84ab28ba0e8644fbe7562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:40:52.347301 31276 raft_consensus.cc:1261] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Refusing update from remote peer 56c29a8a97a84ab28ba0e8644fbe7562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:40:52.347683 31765 consensus_queue.cc:1035] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:40:52.347857 31773 consensus_queue.cc:1035] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:40:52.350602 31780 mvcc.cc:205] Tried to move back new op lower bound from 7315803555211366400 to 7315803555120472064. Current Snapshot: MvccSnapshot[applied={T|T < 7315803555211366400}]
I20260807 06:40:52.352447 31050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:40:52.352548 31050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:40:52.352784 31779 mvcc.cc:205] Tried to move back new op lower bound from 7315803555211366400 to 7315803555120472064. Current Snapshot: MvccSnapshot[applied={T|T < 7315803555211366400}]
I20260807 06:40:52.353353 31050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Sending DeleteTablet for 3 replicas of tablet 0895d8066a0c40a49dc3f206185802ca
I20260807 06:40:52.353694 31251 tablet_service.cc:1514] Processing DeleteTablet for tablet 0895d8066a0c40a49dc3f206185802ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:52 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:53802
I20260807 06:40:52.353878 31632 tablet_service.cc:1514] Processing DeleteTablet for tablet 0895d8066a0c40a49dc3f206185802ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:52 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:58474
I20260807 06:40:52.353983 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 31148
I20260807 06:40:52.353852 31439 tablet_service.cc:1514] Processing DeleteTablet for tablet 0895d8066a0c40a49dc3f206185802ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:52 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:36702
I20260807 06:40:52.353947 31631 tablet_service.cc:1514] Processing DeleteTablet for tablet 0895d8066a0c40a49dc3f206185802ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:52 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:58474
I20260807 06:40:52.354314 31795 tablet_replica.cc:330] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: stopping tablet replica
I20260807 06:40:52.354463 31795 raft_consensus.cc:2229] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:40:52.356328 31795 raft_consensus.cc:2258] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:40:52.357802 31795 ts_tablet_manager.cc:1905] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:40:52.359603 31795 ts_tablet_manager.cc:1918] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:40:52.359754 31795 log.cc:1196] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/0895d8066a0c40a49dc3f206185802ca
I20260807 06:40:52.360074 31795 ts_tablet_manager.cc:1939] T 0895d8066a0c40a49dc3f206185802ca P 56c29a8a97a84ab28ba0e8644fbe7562: Deleting consensus metadata
I20260807 06:40:52.360570 31031 catalog_manager.cc:4841] TS 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839): tablet 0895d8066a0c40a49dc3f206185802ca (table test-table [id=9926611257b0405bb47ae989719283b9]) successfully deleted
I20260807 06:40:52.363827 31794 tablet_replica.cc:330] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: stopping tablet replica
I20260807 06:40:52.363978 31794 raft_consensus.cc:2229] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:40:52.364100 31794 raft_consensus.cc:2258] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:40:52.364609 31794 ts_tablet_manager.cc:1905] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:40:52.365960 31794 ts_tablet_manager.cc:1918] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:40:52.366058 31794 log.cc:1196] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/0895d8066a0c40a49dc3f206185802ca
I20260807 06:40:52.366322 31794 ts_tablet_manager.cc:1939] T 0895d8066a0c40a49dc3f206185802ca P 91a17cc31e224e88b898e8643459f3a0: Deleting consensus metadata
I20260807 06:40:52.366842 31028 catalog_manager.cc:4841] TS 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937): tablet 0895d8066a0c40a49dc3f206185802ca (table test-table [id=9926611257b0405bb47ae989719283b9]) successfully deleted
W20260807 06:40:52.367028 31028 catalog_manager.cc:4808] TS 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937): delete failed for tablet 0895d8066a0c40a49dc3f206185802ca because the tablet was not found. No further retry: Not found: Tablet not found: 0895d8066a0c40a49dc3f206185802ca
W20260807 06:40:52.370373 31031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:44243 (error 108)
I20260807 06:40:52.370972 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.370817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44243 (local address 127.0.0.1:53814)
0807 06:40:52.370876 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.370914 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:44243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:40:52.371079 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 31346
W20260807 06:40:52.376801 31031 catalog_manager.cc:4584] TS 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0895d8066a0c40a49dc3f206185802ca: Network error: Client connection negotiation failed: client connection to 127.11.114.193:44243: connect: Connection refused (error 111)
I20260807 06:40:52.381435 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 31533
I20260807 06:40:52.395186 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 30997
I20260807 06:40:52.406453 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40759
--webserver_interface=127.11.114.254
--webserver_port=35145
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:40759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:52.490370 31798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:52.490594 31798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:52.490617 31798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:52.491901 31798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:52.491943 31798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:52.491958 31798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:52.491971 31798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:52.492007 31798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:52.493561 31798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:40759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=35145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:52.493839 31798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:52.494084 31798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:40:52.503806 31798 server_base.cc:1034] running on GCE node
W20260807 06:40:52.503743 31814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.503729 31809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.503764 31808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:52.504245 31798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:52.504580 31798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:52.505765 31798 hybrid_clock.cc:648] HybridClock initialized: now 1786084852505762 us; error 40 us; skew 500 ppm
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:52.508425 31798 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:40:52.508896 31798 webserver.cc:460] Webserver started at http://127.11.114.254:35145/ using document root <none> and password file <none>
I20260807 06:40:52.509150 31798 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:52.509231 31798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:52.510716 31798 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:52.511394 31827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.511564 31798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:40:52.511696 31798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "ce48457dc21f459db3d90883a78ae502"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "8cbaba356e47c4753e063b25c9226b61"
server_key_iv: "a41056a76eed4ee9671c12ffc068cce8"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.512100 31798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:52.541798 31798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:52.543807 31798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:52.543982 31798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:52.548406 31798 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:40759
I20260807 06:40:52.548441 31897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:40759 every 8 connection(s)
I20260807 06:40:52.548889 31798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:40:52.550779 31898 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:40:52.551420 31898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Bootstrap starting.
I20260807 06:40:52.552469 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 31798
I20260807 06:40:52.552824 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:44243
--local_ip_for_outbound_sockets=127.11.114.193
--tserver_master_addrs=127.11.114.254:40759
--webserver_port=45687
--webserver_interface=127.11.114.193
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:40:52.560477 31898 log.cc:824] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:52.562599 31898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:40:52.562923 31898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Bootstrap complete.
I20260807 06:40:52.565250 31898 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:52.565513 31898 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce48457dc21f459db3d90883a78ae502, State: Initialized, Role: FOLLOWER
I20260807 06:40:52.565750 31898 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:52.565855 31898 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:40:52.565914 31898 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:40:52.565964 31898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:40:52.566980 31898 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:52.567103 31898 leader_election.cc:302] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce48457dc21f459db3d90883a78ae502; no voters:
I20260807 06:40:52.567301 31898 leader_election.cc:288] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:40:52.567377 31908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:40:52.567584 31908 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 2 LEADER]: Becoming Leader. State: Replica: ce48457dc21f459db3d90883a78ae502, State: Running, Role: LEADER
I20260807 06:40:52.567605 31898 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:52.567692 31908 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:52.567952 31909 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ce48457dc21f459db3d90883a78ae502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } } }
I20260807 06:40:52.568043 31909 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:52.567948 31908 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce48457dc21f459db3d90883a78ae502. Latest consensus state: current_term: 2 leader_uuid: "ce48457dc21f459db3d90883a78ae502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } } }
I20260807 06:40:52.568217 31908 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:52.568324 31919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:40:52.569136 31919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:40:52.569239 31919 catalog_manager.cc:1259] Loaded cluster ID: 7def937d07e640b7b37bc26a2f19a286
I20260807 06:40:52.569273 31919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:40:52.569552 31919 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 06:40:52.574770 31919 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 06:40:52.577339 31919 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:40:52.577570 31919 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Loaded TSK: 0
I20260807 06:40:52.577805 31919 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:40:52.644342 31902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:52.644552 31902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:52.644577 31902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:52.645869 31902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:52.645917 31902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:52.645946 31902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:40:52.647404 31902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:44243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=45687
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:52.647655 31902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:52.647918 31902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:52.654142 31943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.654160 31944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.654321 31947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:52.654683 31902 server_base.cc:1034] running on GCE node
I20260807 06:40:52.654887 31902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:52.655189 31902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:52.656392 31902 hybrid_clock.cc:648] HybridClock initialized: now 1786084852656374 us; error 33 us; skew 500 ppm
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:52.659490 31902 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:40:52.660097 31902 webserver.cc:460] Webserver started at http://127.11.114.193:45687/ using document root <none> and password file <none>
I20260807 06:40:52.660353 31902 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:52.660422 31902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:52.662096 31902 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:40:52.662881 31956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.663085 31902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:52.663199 31902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "7a7c2eec402eb13e65ce77daeba8c6e6"
server_key_iv: "d2acdb813c4c719b02ab2e1dc10991ee"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.663580 31902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:52.702785 31902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:52.705036 31902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:52.705233 31902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:52.705502 31902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:52.706014 31967 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:40:52.707515 31902 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:40:52.707604 31902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 06:40:52.707653 31902 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:40:52.708328 31902 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:40:52.708384 31902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 06:40:52.708462 31967 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Bootstrap starting.
I20260807 06:40:52.715983 31902 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:44243
I20260807 06:40:52.716301 32123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:44243 every 8 connection(s)
I20260807 06:40:52.716467 31902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:40:52.717895 31967 log.cc:824] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:52.720008 31967 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:40:52.720304 31967 tablet_bootstrap.cc:492] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Bootstrap complete.
I20260807 06:40:52.721215 31967 ts_tablet_manager.cc:1397] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260807 06:40:52.721560 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 31902
I20260807 06:40:52.721904 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:43839
--local_ip_for_outbound_sockets=127.11.114.194
--tserver_master_addrs=127.11.114.254:40759
--webserver_port=37261
--webserver_interface=127.11.114.194
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:40:52.722481 31967 raft_consensus.cc:348] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.722766 31967 raft_consensus.cc:729] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de0a5f5674d4203bf4b5aeb837dcfc1, State: Initialized, Role: FOLLOWER
I20260807 06:40:52.722903 31967 consensus_queue.cc:260] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } } peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } }
I20260807 06:40:52.723235 31967 ts_tablet_manager.cc:1428] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:52.728705 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.717398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:57819 (local address 127.11.114.254:40759)
0807 06:40:52.717638 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.717647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.717692 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:40:52.718394 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.718405 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.718452 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.718593 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:52.719251 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.720249 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.721315 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.722016 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.722146 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.724915 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:52.724919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:52.724919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:52.724930 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:52.726806 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.727374 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.727377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.727378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.727468 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.727779 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.727780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.727780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.727989 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:52.728055 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.728443 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.728553 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":97,"threads_started":1}
I20260807 06:40:52.728859 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.717277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:57819)
0807 06:40:52.717478 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.717523 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.718163 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.718634 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.718638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.718930 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:52.719123 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.719128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.720360 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.720362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.721210 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.721212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.722265 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.722266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.722412 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.723147 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:52.723159 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:52.724800 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:52.726901 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.726909 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.726912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.727278 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.727539 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.727539 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.727540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.727701 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.728064 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:52.728073 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:52.728267 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.728400 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.728600 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":117,"threads_started":1}
I20260807 06:40:52.729666 32124 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40759
I20260807 06:40:52.729779 32124 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:52.729964 32124 heartbeater.cc:511] Master 127.11.114.254:40759 requested a full tablet report, sending...
I20260807 06:40:52.730810 31850 ts_manager.cc:194] Registered new tserver with Master: 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243)
I20260807 06:40:52.732203 31850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:57819
I20260807 06:40:52.733119 32124 heartbeater.cc:503] Master 127.11.114.254:40759 was elected leader, sending a full tablet report...
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.193@KRBTEST.COM
I20260807 06:40:52.740778 32133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.731623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44243 (local address 127.0.0.1:53828)
0807 06:40:52.731859 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.731885 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.731944 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.732195 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.732201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.732581 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:52.732646 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.732648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.734235 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.734236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.734880 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.734882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.735405 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.735406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.735488 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.736112 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:52.736117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:52.737474 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:52.739485 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.739492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.739494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.739712 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.739978 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.739979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.739979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.740048 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.740359 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:52.740366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:52.740422 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.740518 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.740675 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":94,"threads_started":1}
I20260807 06:40:52.740778 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.731663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53828 (local address 127.11.114.193:44243)
0807 06:40:52.732045 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.732049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.732056 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:40:52.732081 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.732083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.732096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.732174 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:52.733284 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.734143 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.735020 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.735316 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.735404 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.737647 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:52.737655 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:52.737655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:52.737662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:52.739281 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.739833 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.739836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.739836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.739902 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.740120 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.740121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.740122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.740275 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:52.740368 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.740571 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.740673 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":79,"threads_started":1}
I20260807 06:40:52.741299 32031 tablet_service.cc:1514] Processing DeleteTablet for tablet 0895d8066a0c40a49dc3f206185802ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:52 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:53828
I20260807 06:40:52.742017 32136 tablet_replica.cc:330] stopping tablet replica
I20260807 06:40:52.742164 32136 raft_consensus.cc:2229] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:40:52.742244 32136 raft_consensus.cc:2258] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:40:52.742559 32136 ts_tablet_manager.cc:1905] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:40:52.744426 32136 ts_tablet_manager.cc:1918] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:40:52.744514 32136 log.cc:1196] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/0895d8066a0c40a49dc3f206185802ca
I20260807 06:40:52.744801 32136 ts_tablet_manager.cc:1939] T 0895d8066a0c40a49dc3f206185802ca P 4de0a5f5674d4203bf4b5aeb837dcfc1: Deleting consensus metadata
I20260807 06:40:52.745307 31833 catalog_manager.cc:4841] TS 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243): tablet 0895d8066a0c40a49dc3f206185802ca (table test-table [id=9926611257b0405bb47ae989719283b9]) successfully deleted
W20260807 06:40:52.810662 32132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:52.810863 32132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:52.810886 32132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:52.812254 32132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:52.812302 32132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:52.812359 32132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:40:52.813833 32132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:43839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=37261
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:52.814105 32132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:52.814359 32132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:52.819924 32145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.819922 32149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:52.820053 32132 server_base.cc:1034] running on GCE node
W20260807 06:40:52.819900 32146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:52.820331 32132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:52.820607 32132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:52.821754 32132 hybrid_clock.cc:648] HybridClock initialized: now 1786084852821730 us; error 42 us; skew 500 ppm
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:52.824311 32132 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:40:52.824779 32132 webserver.cc:460] Webserver started at http://127.11.114.194:37261/ using document root <none> and password file <none>
I20260807 06:40:52.825040 32132 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:52.825155 32132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:52.826640 32132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:52.827302 32158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.827589 32132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:52.827699 32132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "56c29a8a97a84ab28ba0e8644fbe7562"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "6ad523b8aa1b87d406ed96b47724fddb"
server_key_iv: "04fca9a5772ba1be5580230d902b9f63"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.828042 32132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:52.856173 32132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:52.858559 32132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:52.858737 32132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:52.858992 32132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:52.859324 32132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:52.859380 32132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.859426 32132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:52.859450 32132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.865516 32132 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:43839
I20260807 06:40:52.865574 32313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:43839 every 8 connection(s)
I20260807 06:40:52.865936 32132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:40:52.868410 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 32132
I20260807 06:40:52.868728 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:32937
--local_ip_for_outbound_sockets=127.11.114.195
--tserver_master_addrs=127.11.114.254:40759
--webserver_port=38357
--webserver_interface=127.11.114.195
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:52.876832 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.866693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:38505 (local address 127.11.114.254:40759)
0807 06:40:52.866775 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:52.866777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:52.866980 (+ 203us) server_negotiation.cc:407] Connection header received
0807 06:40:52.867489 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:52.867490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:52.867503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:52.867565 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:52.868889 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.869523 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.870914 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.871262 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.871340 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.873809 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:52.873810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:52.873811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:52.873818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:52.875287 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.875823 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.875824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.875824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.875876 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:52.876175 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:52.876176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:52.876176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:52.876279 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:52.876329 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:52.876697 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 06:40:52.876766 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 06:40:52.877161 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:52.866668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:38505)
0807 06:40:52.866851 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:52.866888 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:52.867400 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:52.867645 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:52.867649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:52.868479 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:52.868777 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.868785 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.869662 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.869663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.870826 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:52.870829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:52.871356 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:52.871357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:52.871468 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:52.872015 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:52.872024 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:52.873712 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:52.875379 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.875386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.875389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.875746 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.875944 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:52.875945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:52.875945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:52.876101 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:52.876349 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:52.876357 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:52.876537 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 06:40:52.876681 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 06:40:52.876873 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260807 06:40:52.877826 32314 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40759
I20260807 06:40:52.877926 32314 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:52.878127 32314 heartbeater.cc:511] Master 127.11.114.254:40759 requested a full tablet report, sending...
I20260807 06:40:52.878575 31850 ts_manager.cc:194] Registered new tserver with Master: 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839)
I20260807 06:40:52.878958 31850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:38505
W20260807 06:40:52.953455 32318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:52.953725 32318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:52.953768 32318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:52.954927 32318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:52.954993 32318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:52.955032 32318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:40:52.956482 32318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:32937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=38357
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:52.956758 32318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:52.956996 32318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:52.962208 32328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:52.962221 32331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:52.962414 32318 server_base.cc:1034] running on GCE node
W20260807 06:40:52.962491 32327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:52.962733 32318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:52.962931 32318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:52.964105 32318 hybrid_clock.cc:648] HybridClock initialized: now 1786084852964088 us; error 47 us; skew 500 ppm
Aug 07 06:40:52 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:52.966710 32318 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:40:52.967234 32318 webserver.cc:460] Webserver started at http://127.11.114.195:38357/ using document root <none> and password file <none>
I20260807 06:40:52.967518 32318 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:52.967623 32318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:52.969117 32318 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:40:52.969767 32344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.969928 32318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:52.970036 32318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "91a17cc31e224e88b898e8643459f3a0"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "e1246299a6ce5475df3c3ad76dd09927"
server_key_iv: "b7e72952ad7d468ae17a12b41dbf4d34"
server_key_version: "encryptionkey@0"
I20260807 06:40:52.970363 32318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:52.991954 32318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:52.994089 32318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:52.994266 32318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:52.994511 32318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:52.994872 32318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:52.994930 32318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:52.994967 32318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:52.994995 32318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.000962 32318 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:32937
I20260807 06:40:53.001004 32506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:32937 every 8 connection(s)
I20260807 06:40:53.001356 32318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:40:53.004945 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 32318
Aug 07 06:40:53 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:40:53.011777 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.002111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:58517 (local address 127.11.114.254:40759)
0807 06:40:53.002165 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:53.002167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:53.002464 (+ 297us) server_negotiation.cc:407] Connection header received
0807 06:40:53.003034 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:53.003036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:53.003050 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:53.003105 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:53.004081 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.004580 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.005845 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.006066 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.006119 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.008586 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:53.008588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:53.008588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:53.008595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:53.010291 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:53.010727 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:53.010729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:53.010729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:53.010793 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:53.011053 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:53.011055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:53.011055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:53.011183 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:53.011264 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:53.011584 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 06:40:53.011674 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:40:53.012137 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.002138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:58517)
0807 06:40:53.002333 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.002369 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:53.002924 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:53.003187 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:53.003191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:53.003779 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:53.003978 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.003983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.004685 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.004686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.005764 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.005767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.006156 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.006157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.006334 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.006936 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:53.006944 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:53.008476 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:53.010391 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:53.010398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:53.010400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:53.010632 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:53.010869 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:53.010870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:53.010870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:53.010984 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:53.011256 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:53.011263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:53.011445 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 06:40:53.011562 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 06:40:53.011744 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260807 06:40:53.012439 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.005417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.11.114.254:40759)
0807 06:40:53.005523 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:53.005525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:53.005675 (+ 150us) server_negotiation.cc:407] Connection header received
0807 06:40:53.005734 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:53.005734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:53.005744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:53.005829 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:53.006571 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.007187 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.008086 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.008359 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.008430 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.009681 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:53.009683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:53.009683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:53.009690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:53.011196 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:53.011575 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:53.011576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:53.011576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:53.011637 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:53.011851 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:53.011852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:53.011852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:53.011974 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:53.012129 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:53.012249 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 06:40:53.012347 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
I20260807 06:40:53.012614 32511 heartbeater.cc:347] Connected to a master server at 127.11.114.254:40759
I20260807 06:40:53.012719 32511 heartbeater.cc:464] Registering TS with master...
I20260807 06:40:53.012917 32511 heartbeater.cc:511] Master 127.11.114.254:40759 requested a full tablet report, sending...
I20260807 06:40:53.013279 31850 ts_manager.cc:194] Registered new tserver with Master: 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937)
I20260807 06:40:53.013753 31850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:58517
I20260807 06:40:53.024153 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:40:53.027056 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.024565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.11.114.254:40759)
0807 06:40:53.024636 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:53.024638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:53.024732 (+ 94us) server_negotiation.cc:407] Connection header received
0807 06:40:53.024793 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:53.024793 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:53.024827 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:53.024888 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:40:53.025002 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.025496 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.026362 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.026554 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.026606 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.026750 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:40:53.026849 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:40:53.026911 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:53.026940 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 06:40:53.026960 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:40:53.027374 31850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:40:53.028993 31850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:40:53.031574 32031 tablet_service.cc:1467] Processing CreateTablet for tablet 5940069a56f041f0909bed4cd9a45330 (DEFAULT_TABLE table=test-table [id=d984190b2be34015a93b1a279bd0924f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:53.031807 32031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5940069a56f041f0909bed4cd9a45330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:53.033182 31967 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Bootstrap starting.
I20260807 06:40:53.033550 31967 tablet_bootstrap.cc:654] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:53.034094 31967 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: No bootstrap required, opened a new log
I20260807 06:40:53.034153 31967 ts_tablet_manager.cc:1397] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:53.034508 31967 raft_consensus.cc:348] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.034638 31967 raft_consensus.cc:374] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:53.034703 31967 raft_consensus.cc:729] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de0a5f5674d4203bf4b5aeb837dcfc1, State: Initialized, Role: FOLLOWER
I20260807 06:40:53.034775 31967 consensus_queue.cc:260] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.034884 31967 ts_tablet_manager.cc:1428] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 06:40:53.034950 32133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.031093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43839 (local address 127.0.0.1:36714)
0807 06:40:53.031154 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.031166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:53.031206 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:53.031618 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:53.031619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:53.031620 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:53.031663 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.031664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.032619 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.032620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.033578 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.033582 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.034610 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.034611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.034691 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.034714 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:40:53.034785 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:40:53.034832 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:40:53.035130 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.031228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36714 (local address 127.11.114.194:43839)
0807 06:40:53.031452 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:53.031456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:53.031466 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:40:53.031504 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:53.031511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:53.031524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:53.031614 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:53.031730 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.032518 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.033686 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.034509 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.034597 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.034678 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:53.034839 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 06:40:53.035044 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":116,"threads_started":1}
I20260807 06:40:53.035835 32226 tablet_service.cc:1467] Processing CreateTablet for tablet 5940069a56f041f0909bed4cd9a45330 (DEFAULT_TABLE table=test-table [id=d984190b2be34015a93b1a279bd0924f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:53.036239 32226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5940069a56f041f0909bed4cd9a45330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:53.037278 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.031215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:32937 (local address 127.0.0.1:58484)
0807 06:40:53.031320 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.031329 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:53.031397 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:53.033776 (+ 2379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:53.033777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:53.033779 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:53.033825 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.033827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.034967 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.034968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.036145 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.036148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.037064 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.037065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.037114 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.037118 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:40:53.037193 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 06:40:53.037226 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":74,"threads_started":1}
I20260807 06:40:53.037330 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.033237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.11.114.195:32937)
0807 06:40:53.033455 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:53.033460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:53.033472 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:40:53.033516 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:53.033518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:53.033572 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:53.033697 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:53.033927 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.034853 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.036230 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.036958 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.037042 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.037094 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:53.037210 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 06:40:53.037251 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
I20260807 06:40:53.038333 32408 tablet_service.cc:1467] Processing CreateTablet for tablet 5940069a56f041f0909bed4cd9a45330 (DEFAULT_TABLE table=test-table [id=d984190b2be34015a93b1a279bd0924f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:40:53.038698 32408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5940069a56f041f0909bed4cd9a45330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:40:53.039863 32524 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: Bootstrap starting.
I20260807 06:40:53.040479 32524 tablet_bootstrap.cc:654] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:53.041044 32524 log.cc:824] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:53.041991 32524 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: No bootstrap required, opened a new log
I20260807 06:40:53.042074 32524 ts_tablet_manager.cc:1397] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:40:53.042125 32528 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: Bootstrap starting.
I20260807 06:40:53.042768 32528 tablet_bootstrap.cc:654] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: Neither blocks nor log segments found. Creating new log.
I20260807 06:40:53.043368 32528 log.cc:824] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:53.044167 32528 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: No bootstrap required, opened a new log
I20260807 06:40:53.044243 32528 ts_tablet_manager.cc:1397] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:40:53.044273 32524 raft_consensus.cc:348] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.044454 32524 raft_consensus.cc:374] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:53.044490 32524 raft_consensus.cc:729] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c29a8a97a84ab28ba0e8644fbe7562, State: Initialized, Role: FOLLOWER
I20260807 06:40:53.044612 32524 consensus_queue.cc:260] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.044919 32524 ts_tablet_manager.cc:1428] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:40:53.045071 32314 heartbeater.cc:503] Master 127.11.114.254:40759 was elected leader, sending a full tablet report...
I20260807 06:40:53.046283 32528 raft_consensus.cc:348] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.046402 32528 raft_consensus.cc:374] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:40:53.046423 32528 raft_consensus.cc:729] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a17cc31e224e88b898e8643459f3a0, State: Initialized, Role: FOLLOWER
I20260807 06:40:53.046525 32528 consensus_queue.cc:260] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.046758 32528 ts_tablet_manager.cc:1428] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:40:53.046820 32511 heartbeater.cc:503] Master 127.11.114.254:40759 was elected leader, sending a full tablet report...
I20260807 06:40:53.063951 32530 raft_consensus.cc:482] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:40:53.064069 32530 raft_consensus.cc:504] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.064404 32530 leader_election.cc:288] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937), 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243)
I20260807 06:40:53.068632 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.064612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44243 (local address 127.11.114.194:42541)
0807 06:40:53.064809 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.064824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:53.064883 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:53.065062 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:53.065063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:53.065065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:53.065131 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.065132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.066066 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.066067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.067485 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.067487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.068323 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.068324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.068416 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.068421 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:40:53.068493 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 06:40:53.068540 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":97,"thread_start_us":112,"threads_started":1}
I20260807 06:40:53.068671 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.064668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:42541 (local address 127.11.114.193:44243)
0807 06:40:53.064724 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:53.064727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:53.064907 (+ 180us) server_negotiation.cc:407] Connection header received
0807 06:40:53.064959 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:53.064960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:53.064988 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:53.065042 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:53.065214 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.065977 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.067577 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.068223 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.068302 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.068393 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:53.068503 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 06:40:53.068588 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 06:40:53.068715 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.064760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:39075 (local address 127.11.114.195:32937)
0807 06:40:53.064817 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:53.064819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:53.064826 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:40:53.064847 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:53.064848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:53.064856 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:53.064931 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:40:53.065145 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.065977 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.067562 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.068171 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.068282 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.068295 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:53.068600 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 06:40:53.068645 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:40:53.069077 32059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5940069a56f041f0909bed4cd9a45330" candidate_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" is_pre_election: true
I20260807 06:40:53.069242 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.064612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:32937 (local address 127.11.114.194:39075)
0807 06:40:53.064670 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.064683 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:53.064728 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:53.064954 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:53.064955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:53.064956 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:40:53.065043 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.065046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.066066 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.066067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.067450 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:53.067453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.068432 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.068433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:53.068513 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.068515 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:40:53.068567 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:40:53.068601 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":59}
I20260807 06:40:53.069245 32059 raft_consensus.cc:2454] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56c29a8a97a84ab28ba0e8644fbe7562 in term 0.
I20260807 06:40:53.069579 32164 leader_election.cc:302] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de0a5f5674d4203bf4b5aeb837dcfc1, 56c29a8a97a84ab28ba0e8644fbe7562; no voters:
I20260807 06:40:53.069629 32440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5940069a56f041f0909bed4cd9a45330" candidate_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a17cc31e224e88b898e8643459f3a0" is_pre_election: true
I20260807 06:40:53.069730 32530 raft_consensus.cc:2781] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:40:53.069787 32530 raft_consensus.cc:482] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:40:53.069808 32530 raft_consensus.cc:3037] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:53.069813 32440 raft_consensus.cc:2454] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56c29a8a97a84ab28ba0e8644fbe7562 in term 0.
I20260807 06:40:53.070551 32530 raft_consensus.cc:504] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.070700 32530 leader_election.cc:288] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [CANDIDATE]: Term 1 election: Requested vote from peers 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937), 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243)
I20260807 06:40:53.070850 32059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5940069a56f041f0909bed4cd9a45330" candidate_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1"
I20260807 06:40:53.070842 32440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5940069a56f041f0909bed4cd9a45330" candidate_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a17cc31e224e88b898e8643459f3a0"
I20260807 06:40:53.070927 32059 raft_consensus.cc:3037] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:53.070928 32440 raft_consensus.cc:3037] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:40:53.071821 32059 raft_consensus.cc:2454] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56c29a8a97a84ab28ba0e8644fbe7562 in term 1.
I20260807 06:40:53.071874 32440 raft_consensus.cc:2454] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56c29a8a97a84ab28ba0e8644fbe7562 in term 1.
I20260807 06:40:53.072047 32164 leader_election.cc:302] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de0a5f5674d4203bf4b5aeb837dcfc1, 56c29a8a97a84ab28ba0e8644fbe7562; no voters:
I20260807 06:40:53.072206 32530 raft_consensus.cc:2781] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:40:53.072402 32530 raft_consensus.cc:686] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 LEADER]: Becoming Leader. State: Replica: 56c29a8a97a84ab28ba0e8644fbe7562, State: Running, Role: LEADER
I20260807 06:40:53.072508 32530 consensus_queue.cc:237] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.073498 31850 catalog_manager.cc:5495] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } health_report { overall_health: UNKNOWN } } }
I20260807 06:40:53.083424 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.081061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.11.114.194:43839)
0807 06:40:53.081125 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:53.081127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:53.081137 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:40:53.081193 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:53.081194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:53.081220 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:53.081295 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:40:53.081406 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.081937 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.082705 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:53.082892 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:53.082955 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:53.083101 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:40:53.083199 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:40:53.083266 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:53.083301 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 06:40:53.083321 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:40:53.085332 32059 raft_consensus.cc:1261] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Refusing update from remote peer 56c29a8a97a84ab28ba0e8644fbe7562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:40:53.085353 32440 raft_consensus.cc:1261] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 1 FOLLOWER]: Refusing update from remote peer 56c29a8a97a84ab28ba0e8644fbe7562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:40:53.085723 32535 consensus_queue.cc:1035] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:40:53.085865 32530 consensus_queue.cc:1035] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:40:53.088213 32539 mvcc.cc:205] Tried to move back new op lower bound from 7315803558234644480 to 7315803558185795584. Current Snapshot: MvccSnapshot[applied={T|T < 7315803558234644480}]
I20260807 06:40:53.091007 31849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:40:53.091109 31849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:40:53.091907 31849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Sending DeleteTablet for 3 replicas of tablet 5940069a56f041f0909bed4cd9a45330
I20260807 06:40:53.092242 32226 tablet_service.cc:1514] Processing DeleteTablet for tablet 5940069a56f041f0909bed4cd9a45330 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:53 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:36714
I20260807 06:40:53.092262 32408 tablet_service.cc:1514] Processing DeleteTablet for tablet 5940069a56f041f0909bed4cd9a45330 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:53 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:58484
I20260807 06:40:53.092252 32031 tablet_service.cc:1514] Processing DeleteTablet for tablet 5940069a56f041f0909bed4cd9a45330 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:40:53 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:53828
I20260807 06:40:53.092298 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 31902
I20260807 06:40:53.092453 32136 tablet_replica.cc:330] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: stopping tablet replica
I20260807 06:40:53.092484 32553 tablet_replica.cc:330] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: stopping tablet replica
I20260807 06:40:53.092514 32136 raft_consensus.cc:2229] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:40:53.092566 32136 raft_consensus.cc:2258] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:40:53.092566 32553 raft_consensus.cc:2229] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:40:53.092658 32553 raft_consensus.cc:2258] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:40:53.094743 32553 ts_tablet_manager.cc:1905] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:40:53.096366 32554 tablet_replica.cc:330] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: stopping tablet replica
I20260807 06:40:53.096504 32554 raft_consensus.cc:2229] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:40:53.096544 32553 ts_tablet_manager.cc:1918] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:40:53.096619 32554 raft_consensus.cc:2258] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:40:53.096632 32553 log.cc:1196] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/5940069a56f041f0909bed4cd9a45330
I20260807 06:40:53.097044 32553 ts_tablet_manager.cc:1939] T 5940069a56f041f0909bed4cd9a45330 P 91a17cc31e224e88b898e8643459f3a0: Deleting consensus metadata
I20260807 06:40:53.097608 31831 catalog_manager.cc:4841] TS 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937): tablet 5940069a56f041f0909bed4cd9a45330 (table test-table [id=d984190b2be34015a93b1a279bd0924f]) successfully deleted
I20260807 06:40:53.098949 32554 ts_tablet_manager.cc:1905] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:40:53.100545 32554 ts_tablet_manager.cc:1918] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:40:53.100615 32554 log.cc:1196] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/5940069a56f041f0909bed4cd9a45330
I20260807 06:40:53.100904 32554 ts_tablet_manager.cc:1939] T 5940069a56f041f0909bed4cd9a45330 P 56c29a8a97a84ab28ba0e8644fbe7562: Deleting consensus metadata
I20260807 06:40:53.101392 31833 catalog_manager.cc:4841] TS 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839): tablet 5940069a56f041f0909bed4cd9a45330 (table test-table [id=d984190b2be34015a93b1a279bd0924f]) successfully deleted
W20260807 06:40:53.108152 31833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:44243 (error 108)
I20260807 06:40:53.108611 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 32132
I20260807 06:40:53.113446 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.113219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:44243 (local address 127.0.0.1:53844)
0807 06:40:53.113322 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.113394 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:44243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 06:40:53.114135 31833 catalog_manager.cc:4584] TS 4de0a5f5674d4203bf4b5aeb837dcfc1 (127.11.114.193:44243): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5940069a56f041f0909bed4cd9a45330: Network error: Client connection negotiation failed: client connection to 127.11.114.193:44243: connect: Connection refused (error 111)
I20260807 06:40:53.118880 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 32318
I20260807 06:40:53.131104 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 31798
I20260807 06:40:53.140005 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40759
--webserver_interface=127.11.114.254
--webserver_port=35145
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:40759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:53.222893 32557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:53.223152 32557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:53.223193 32557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:53.224332 32557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:40:53.224403 32557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:53.224428 32557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:53.224449 32557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:40:53.224478 32557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:40:53.226008 32557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:40759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=35145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:53.226297 32557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:53.226538 32557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:53.231498 32565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:53.231491 32563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:53.231491 32568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:53.231827 32557 server_base.cc:1034] running on GCE node
I20260807 06:40:53.232102 32557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:53.232412 32557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:53.233587 32557 hybrid_clock.cc:648] HybridClock initialized: now 1786084853233558 us; error 35 us; skew 500 ppm
Aug 07 06:40:53 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:53.236212 32557 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:40:53.236685 32557 webserver.cc:460] Webserver started at http://127.11.114.254:35145/ using document root <none> and password file <none>
I20260807 06:40:53.236896 32557 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:53.236937 32557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:53.238307 32557 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:40:53.238991 32584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.239244 32557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:40:53.239339 32557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "ce48457dc21f459db3d90883a78ae502"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "8cbaba356e47c4753e063b25c9226b61"
server_key_iv: "a41056a76eed4ee9671c12ffc068cce8"
server_key_version: "encryptionkey@0"
I20260807 06:40:53.239681 32557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:53.257068 32557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:53.259373 32557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:53.259550 32557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:53.263878 32557 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:40759
I20260807 06:40:53.263969 32658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:40759 every 8 connection(s)
I20260807 06:40:53.264328 32557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:40:53.265324 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 32557
I20260807 06:40:53.265738 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:44243
--local_ip_for_outbound_sockets=127.11.114.193
--tserver_master_addrs=127.11.114.254:40759
--webserver_port=45687
--webserver_interface=127.11.114.193
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:40:53.266717 32659 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:40:53.267751 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Bootstrap starting.
I20260807 06:40:53.277904 32659 log.cc:824] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:53.281031 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:40:53.281397 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502: Bootstrap complete.
I20260807 06:40:53.284754 32659 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:53.285145 32659 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce48457dc21f459db3d90883a78ae502, State: Initialized, Role: FOLLOWER
I20260807 06:40:53.285329 32659 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:53.285403 32659 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:40:53.285449 32659 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:40:53.285494 32659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:40:53.286835 32659 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:53.286947 32659 leader_election.cc:302] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce48457dc21f459db3d90883a78ae502; no voters:
I20260807 06:40:53.287122 32659 leader_election.cc:288] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 06:40:53.287220 32668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:40:53.287426 32668 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [term 3 LEADER]: Becoming Leader. State: Replica: ce48457dc21f459db3d90883a78ae502, State: Running, Role: LEADER
I20260807 06:40:53.287431 32659 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:40:53.287549 32668 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } }
I20260807 06:40:53.287819 32668 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce48457dc21f459db3d90883a78ae502. Latest consensus state: current_term: 3 leader_uuid: "ce48457dc21f459db3d90883a78ae502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } } }
I20260807 06:40:53.287904 32668 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:53.288076 32669 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ce48457dc21f459db3d90883a78ae502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce48457dc21f459db3d90883a78ae502" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40759 } } }
I20260807 06:40:53.288144 32669 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce48457dc21f459db3d90883a78ae502 [sys.catalog]: This master's current role is: LEADER
I20260807 06:40:53.288496 32675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:40:53.289366 32675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:40:53.289577 32675 catalog_manager.cc:1259] Loaded cluster ID: 7def937d07e640b7b37bc26a2f19a286
I20260807 06:40:53.289779 32675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 06:40:53.290155 32675 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0eb32179ad google::LogMessage::Fail() at ??:0
@ 0x7f0eb321a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0eb32174de google::LogMessage::Flush() at ??:0
@ 0x7f0eb3218a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0ebb1380f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f0ebb1418b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0eb50e50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0eb50da954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0eb5f3e6db start_thread at ??:0
@ 0x7f0eb1d1561f clone at ??:0
W20260807 06:40:53.371886 32666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:53.372087 32666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:53.372112 32666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:53.373260 32666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:53.373298 32666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:53.373324 32666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:40:53.374835 32666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:44243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=45687
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:53.375087 32666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:53.375341 32666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:53.381747 32716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:53.381963 32666 server_base.cc:1034] running on GCE node
W20260807 06:40:53.382109 32715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:53.382141 32721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:53.382429 32666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:53.382733 32666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:53.383895 32666 hybrid_clock.cc:648] HybridClock initialized: now 1786084853383884 us; error 28 us; skew 500 ppm
Aug 07 06:40:53 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:53.386523 32666 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:40:53.387029 32666 webserver.cc:460] Webserver started at http://127.11.114.193:45687/ using document root <none> and password file <none>
I20260807 06:40:53.387277 32666 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:53.387323 32666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:53.388729 32666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:40:53.389475 32731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.389663 32666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:40:53.389773 32666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1"
format_stamp: "Formatted at 2026-08-07 06:40:51 on dist-test-slave-gkw7"
server_key: "7a7c2eec402eb13e65ce77daeba8c6e6"
server_key_iv: "d2acdb813c4c719b02ab2e1dc10991ee"
server_key_version: "encryptionkey@0"
I20260807 06:40:53.390118 32666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:53.404635 32666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:53.407184 32666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:53.407338 32666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:53.407585 32666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:53.408113 32741 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:40:53.409725 32666 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:40:53.409786 32666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 06:40:53.409822 32666 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:40:53.410697 32666 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:40:53.410753 32666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 06:40:53.410823 32741 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Bootstrap starting.
I20260807 06:40:53.418519 32666 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:44243
I20260807 06:40:53.418968 32666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:40:53.421695 32741 log.cc:824] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Log is configured to *not* fsync() on all Append() calls
I20260807 06:40:53.421734 425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:44243 every 8 connection(s)
I20260807 06:40:53.422184 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 32666
I20260807 06:40:53.422523 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:43839
--local_ip_for_outbound_sockets=127.11.114.194
--tserver_master_addrs=127.11.114.254:40759
--webserver_port=37261
--webserver_interface=127.11.114.194
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:40:53.424669 32741 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:40:53.424957 32741 tablet_bootstrap.cc:492] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Bootstrap complete.
I20260807 06:40:53.426036 32741 ts_tablet_manager.cc:1397] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.010s
I20260807 06:40:53.427934 32741 raft_consensus.cc:348] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.428339 32741 raft_consensus.cc:729] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de0a5f5674d4203bf4b5aeb837dcfc1, State: Initialized, Role: FOLLOWER
I20260807 06:40:53.428531 32741 consensus_queue.cc:260] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:53.428864 32741 ts_tablet_manager.cc:1428] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20260807 06:40:53.478610 431 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:40:53.419740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:59951)
0807 06:40:53.419966 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.420030 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:53.420778 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:53.478402 (+ 57624us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
W20260807 06:40:53.478917 426 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:40759 (0 consecutive failures): Network error: Failed to ping master at 127.11.114.254:40759: Client connection negotiation failed: client connection to 127.11.114.254:40759: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:40:53.479313 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.479184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:57165)
0807 06:40:53.479244 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.479289 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 06:40:53.479698 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.479594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:36237)
0807 06:40:53.479642 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.479678 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:40:53.479823 426 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:40:53.511308 436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:53.511519 436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:53.511543 436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:53.512730 436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:53.512796 436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:53.512842 436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:40:53.514451 436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:43839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=37261
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:53.514736 436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:53.514968 436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:53.520490 446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:53.520490 445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:53.520478 448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:53.520548 436 server_base.cc:1034] running on GCE node
I20260807 06:40:53.520967 436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:53.521189 436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:53.522334 436 hybrid_clock.cc:648] HybridClock initialized: now 1786084853522323 us; error 33 us; skew 500 ppm
Aug 07 06:40:53 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:53.524948 436 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:40:53.525403 436 webserver.cc:460] Webserver started at http://127.11.114.194:37261/ using document root <none> and password file <none>
I20260807 06:40:53.525681 436 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:53.525730 436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:53.527032 436 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:40:53.527616 456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.527813 436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:40:53.527908 436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "56c29a8a97a84ab28ba0e8644fbe7562"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "6ad523b8aa1b87d406ed96b47724fddb"
server_key_iv: "04fca9a5772ba1be5580230d902b9f63"
server_key_version: "encryptionkey@0"
I20260807 06:40:53.528246 436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:53.539966 436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:53.542476 436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:53.542634 436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:53.542898 436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:53.543247 436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:53.543304 436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.543346 436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:53.543375 436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.549671 436 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:43839
I20260807 06:40:53.549707 621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:43839 every 8 connection(s)
I20260807 06:40:53.550112 436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:40:53.551254 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.550863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:58329)
0807 06:40:53.551027 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.551108 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
W20260807 06:40:53.551715 622 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:40759 (0 consecutive failures): Network error: Failed to ping master at 127.11.114.254:40759: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
I20260807 06:40:53.552024 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.551915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:40985)
0807 06:40:53.551978 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.552006 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 06:40:53.552340 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.552263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:33717)
0807 06:40:53.552304 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.552326 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 06:40:53.552461 622 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:40:53.559417 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 436
I20260807 06:40:53.559664 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:32937
--local_ip_for_outbound_sockets=127.11.114.195
--tserver_master_addrs=127.11.114.254:40759
--webserver_port=38357
--webserver_interface=127.11.114.195
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:40:53.645569 626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:40:53.645789 626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:40:53.645815 626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:40:53.646999 626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:40:53.647033 626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:40:53.647059 626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:40:53.648433 626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:32937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=38357
--tserver_master_addrs=127.11.114.254:40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:40:53.648617 626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:40:53.648793 626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:40:53.653868 632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:40:53.653898 637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:53.654137 626 server_base.cc:1034] running on GCE node
W20260807 06:40:53.653908 633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:40:53.654382 626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:40:53.654673 626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:40:53.655838 626 hybrid_clock.cc:648] HybridClock initialized: now 1786084853655832 us; error 40 us; skew 500 ppm
Aug 07 06:40:53 dist-test-slave-gkw7 krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:40:53.658489 626 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:40:53.658878 626 webserver.cc:460] Webserver started at http://127.11.114.195:38357/ using document root <none> and password file <none>
I20260807 06:40:53.659060 626 fs_manager.cc:359] Metadata directory not provided
I20260807 06:40:53.659108 626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:40:53.660377 626 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:40:53.660960 645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.661230 626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:40:53.661346 626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "91a17cc31e224e88b898e8643459f3a0"
format_stamp: "Formatted at 2026-08-07 06:40:52 on dist-test-slave-gkw7"
server_key: "e1246299a6ce5475df3c3ad76dd09927"
server_key_iv: "b7e72952ad7d468ae17a12b41dbf4d34"
server_key_version: "encryptionkey@0"
I20260807 06:40:53.661806 626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:40:53.689009 626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:40:53.691372 626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:40:53.691581 626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:40:53.691877 626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:40:53.692242 626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:40:53.692279 626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.692332 626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:40:53.692373 626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:40:53.698705 626 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:32937
I20260807 06:40:53.698748 804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:32937 every 8 connection(s)
I20260807 06:40:53.699146 626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:40:53.700280 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.699892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:55667)
0807 06:40:53.700056 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.700135 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
W20260807 06:40:53.700721 806 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:40759 (0 consecutive failures): Network error: Failed to ping master at 127.11.114.254:40759: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
I20260807 06:40:53.701041 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.700907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:47495)
0807 06:40:53.700977 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.701012 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 06:40:53.701390 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:53.701297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:34361)
0807 06:40:53.701343 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:53.701371 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:40:53.701556 806 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:40:53.705984 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 626
I20260807 06:40:54.480466 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:54.480217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:44657)
0807 06:40:54.480354 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:54.480393 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260807 06:40:54.553050 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:54.552826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:38447)
0807 06:40:54.552955 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:54.552996 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260807 06:40:54.702219 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:54.701921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:33705)
0807 06:40:54.702057 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:54.702110 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260807 06:40:54.778239 815 raft_consensus.cc:482] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:40:54.778409 815 raft_consensus.cc:504] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:54.778781 815 leader_election.cc:288] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839), 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937)
Aug 07 06:40:54 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.195@KRBTEST.COM
Aug 07 06:40:54 dist-test-slave-gkw7 krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.194@KRBTEST.COM
I20260807 06:40:54.789604 819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:54.779136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:35551 (local address 127.11.114.195:32937)
0807 06:40:54.779305 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:54.779315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:54.779357 (+ 42us) server_negotiation.cc:407] Connection header received
0807 06:40:54.779794 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:54.779796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:54.779838 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:54.779946 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:54.780739 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:54.781657 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:54.782640 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:54.783064 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:54.783185 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:54.785704 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:54.785709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:54.785709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:54.785720 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:54.787575 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:54.788323 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:54.788325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:54.788326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:54.788408 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:54.788732 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:54.788733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:54.788734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:54.788967 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:54.789059 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:54.789335 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 06:40:54.789463 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260807 06:40:54.789628 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:54.779135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:45177 (local address 127.11.114.194:43839)
0807 06:40:54.779303 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 06:40:54.779316 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:40:54.779354 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:40:54.779793 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:40:54.779796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:40:54.779838 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:40:54.779996 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:40:54.780734 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:54.781649 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:54.782658 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:40:54.783068 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:40:54.783185 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:54.785904 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:40:54.785908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:40:54.785908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:40:54.785918 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:40:54.787967 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:54.788312 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:54.788315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:54.788315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:54.788399 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:40:54.788685 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:40:54.788686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:40:54.788687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:40:54.788923 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:40:54.789021 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 06:40:54.789338 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 06:40:54.789463 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":57,"threads_started":1}
I20260807 06:40:54.789593 816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:54.778990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:32937 (local address 127.11.114.193:35551)
0807 06:40:54.779242 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:54.779253 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:54.779314 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:54.780051 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:54.780052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:54.780419 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:54.780602 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:54.780608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:54.781791 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:54.781792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:54.782545 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:54.782547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:54.783175 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:54.783176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:54.783286 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:54.783941 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:54.783945 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:54.785473 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:54.787662 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:54.787670 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:54.787672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:54.787979 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:54.788494 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:54.788495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:54.788496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:54.788663 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:54.789053 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:54.789054 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:54.789168 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 06:40:54.789276 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 06:40:54.789476 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":204,"thread_start_us":73,"threads_started":1}
I20260807 06:40:54.789861 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:54.778990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43839 (local address 127.11.114.193:45177)
0807 06:40:54.779059 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:54.779074 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:40:54.779184 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:40:54.780024 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:40:54.780026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:40:54.780418 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:40:54.780602 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:54.780608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:54.781749 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:54.781750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:54.782571 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:40:54.782574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:40:54.783172 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:40:54.783174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:40:54.783289 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:40:54.783893 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:40:54.783901 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:40:54.785781 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:40:54.788054 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:54.788054 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:54.788055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:54.788241 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:54.788485 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:40:54.788486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:40:54.788486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:40:54.788615 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:40:54.789001 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:40:54.789007 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:40:54.789172 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 06:40:54.789277 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 06:40:54.789587 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":127}
W20260807 06:40:54.790701 32737 leader_election.cc:341] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839): Not found: Tablet not found: 5940069a56f041f0909bed4cd9a45330
W20260807 06:40:54.790838 32734 leader_election.cc:341] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937): Not found: Tablet not found: 5940069a56f041f0909bed4cd9a45330
I20260807 06:40:54.790896 32734 leader_election.cc:302] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4de0a5f5674d4203bf4b5aeb837dcfc1; no voters: 56c29a8a97a84ab28ba0e8644fbe7562, 91a17cc31e224e88b898e8643459f3a0
I20260807 06:40:54.791002 815 raft_consensus.cc:2726] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:40:55.481379 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:55.481073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:59717)
0807 06:40:55.481274 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:55.481328 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":88,"threads_started":1}
I20260807 06:40:55.553753 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:55.553546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:52939)
0807 06:40:55.553665 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:55.553709 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260807 06:40:55.702941 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:55.702717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:57295)
0807 06:40:55.702862 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:55.702901 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260807 06:40:56.482169 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:56.481931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:57211)
0807 06:40:56.482077 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:56.482115 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260807 06:40:56.554605 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:56.554326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:37389)
0807 06:40:56.554502 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:56.554556 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
I20260807 06:40:56.680294 833 raft_consensus.cc:482] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:40:56.680398 833 raft_consensus.cc:504] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:40:56.680570 833 leader_election.cc:288] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839), 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937)
W20260807 06:40:56.681032 32737 leader_election.cc:341] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839): Not found: Tablet not found: 5940069a56f041f0909bed4cd9a45330
W20260807 06:40:56.681156 32734 leader_election.cc:341] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937): Not found: Tablet not found: 5940069a56f041f0909bed4cd9a45330
I20260807 06:40:56.681239 32734 leader_election.cc:302] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4de0a5f5674d4203bf4b5aeb837dcfc1; no voters: 56c29a8a97a84ab28ba0e8644fbe7562, 91a17cc31e224e88b898e8643459f3a0
I20260807 06:40:56.681335 833 raft_consensus.cc:2726] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:40:56.703650 834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:56.703431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:59479)
0807 06:40:56.703576 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:56.703610 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260807 06:40:57.483026 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:57.482761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:59537)
0807 06:40:57.482926 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:57.482976 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":67,"threads_started":1}
I20260807 06:40:57.555511 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:57.555285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:45183)
0807 06:40:57.555421 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:57.555466 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260807 06:40:57.704499 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:57.704240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:56761)
0807 06:40:57.704412 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:57.704455 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260807 06:40:58.483930 838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:58.483685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:44189)
0807 06:40:58.483842 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:58.483882 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260807 06:40:58.556284 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:58.556042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:59253)
0807 06:40:58.556200 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:58.556242 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260807 06:40:58.705271 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:58.705021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:37349)
0807 06:40:58.705189 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:58.705229 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
I20260807 06:40:59.484622 842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:59.484394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:41309)
0807 06:40:59.484539 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:59.484576 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260807 06:40:59.557054 843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:59.556797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:53171)
0807 06:40:59.556948 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:59.556991 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260807 06:40:59.705963 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:40:59.705753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:45715)
0807 06:40:59.705880 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 06:40:59.705918 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260807 06:41:00.008644 846 raft_consensus.cc:482] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:41:00.008749 846 raft_consensus.cc:504] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:41:00.009016 846 leader_election.cc:288] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839), 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937)
W20260807 06:41:00.009467 32734 leader_election.cc:341] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937): Not found: Tablet not found: 5940069a56f041f0909bed4cd9a45330
W20260807 06:41:00.009622 32737 leader_election.cc:341] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839): Not found: Tablet not found: 5940069a56f041f0909bed4cd9a45330
I20260807 06:41:00.009698 32737 leader_election.cc:302] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4de0a5f5674d4203bf4b5aeb837dcfc1; no voters: 56c29a8a97a84ab28ba0e8644fbe7562, 91a17cc31e224e88b898e8643459f3a0
I20260807 06:41:00.009795 846 raft_consensus.cc:2726] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:41:00.485404 847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:00.485162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:47085)
0807 06:41:00.485315 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:00.485355 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
I20260807 06:41:00.557840 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:00.557617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:52821)
0807 06:41:00.557762 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:00.557800 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260807 06:41:00.706811 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:00.706574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:55993)
0807 06:41:00.706728 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:00.706765 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260807 06:41:01.486338 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:01.486055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:51341)
0807 06:41:01.486241 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:01.486288 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260807 06:41:01.558681 851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:01.558413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:33687)
0807 06:41:01.558569 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:01.558635 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260807 06:41:01.707679 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:01.707440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:35149)
0807 06:41:01.707584 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:01.707630 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260807 06:41:02.487159 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:02.486909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:60083)
0807 06:41:02.487071 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:02.487113 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260807 06:41:02.559540 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:02.559301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:49431)
0807 06:41:02.559453 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:02.559495 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260807 06:41:02.594996 856 raft_consensus.cc:482] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:41:02.595095 856 raft_consensus.cc:504] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56c29a8a97a84ab28ba0e8644fbe7562" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43839 } } peers { permanent_uuid: "91a17cc31e224e88b898e8643459f3a0" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 32937 } } peers { permanent_uuid: "4de0a5f5674d4203bf4b5aeb837dcfc1" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 44243 } }
I20260807 06:41:02.595263 856 leader_election.cc:288] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839), 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937)
W20260807 06:41:02.595770 32734 leader_election.cc:341] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91a17cc31e224e88b898e8643459f3a0 (127.11.114.195:32937): Not found: Tablet not found: 5940069a56f041f0909bed4cd9a45330
W20260807 06:41:02.595865 32737 leader_election.cc:341] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56c29a8a97a84ab28ba0e8644fbe7562 (127.11.114.194:43839): Not found: Tablet not found: 5940069a56f041f0909bed4cd9a45330
I20260807 06:41:02.595911 32737 leader_election.cc:302] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4de0a5f5674d4203bf4b5aeb837dcfc1; no voters: 56c29a8a97a84ab28ba0e8644fbe7562, 91a17cc31e224e88b898e8643459f3a0
I20260807 06:41:02.596009 856 raft_consensus.cc:2726] T 5940069a56f041f0909bed4cd9a45330 P 4de0a5f5674d4203bf4b5aeb837dcfc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:41:02.708431 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:02.708191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:49655)
0807 06:41:02.708335 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:02.708372 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260807 06:41:03.487937 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:03.487691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:33825)
0807 06:41:03.487852 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:03.487891 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
I20260807 06:41:03.560284 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:03.560062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:41765)
0807 06:41:03.560198 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:03.560240 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260807 06:41:03.709137 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:03.708914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:42701)
0807 06:41:03.709054 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:03.709089 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260807 06:41:04.488631 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:04.488407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:37987)
0807 06:41:04.488548 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:04.488584 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260807 06:41:04.561158 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:04.560838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:53407)
0807 06:41:04.561065 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:04.561108 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":120,"threads_started":1}
I20260807 06:41:04.709960 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:04.709721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:38037)
0807 06:41:04.709882 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:04.709917 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260807 06:41:05.489450 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:05.489195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:60849)
0807 06:41:05.489359 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:05.489400 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":68,"threads_started":1}
I20260807 06:41:05.562145 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:05.561825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:47633)
0807 06:41:05.562040 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:05.562087 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260807 06:41:05.710747 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:05.710514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:53981)
0807 06:41:05.710666 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:05.710700 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260807 06:41:06.490324 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:06.490025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:55837)
0807 06:41:06.490206 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:06.490265 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
I20260807 06:41:06.562959 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:06.562637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:56069)
0807 06:41:06.562867 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:06.562912 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":125,"threads_started":1}
I20260807 06:41:06.711517 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:06.711253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:51611)
0807 06:41:06.711422 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:06.711466 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260807 06:41:07.491178 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:07.490904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:50819)
0807 06:41:07.491055 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:07.491097 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260807 06:41:07.563771 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:07.563522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:43333)
0807 06:41:07.563683 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:07.563722 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260807 06:41:07.712288 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:07.712033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:44229)
0807 06:41:07.712183 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:07.712222 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260807 06:41:08.492178 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:08.491924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.193:57909)
0807 06:41:08.492096 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:08.492132 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260807 06:41:08.564846 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:08.564508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.194:37577)
0807 06:41:08.564733 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:08.564794 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":119,"threads_started":1}
I20260807 06:41:08.712965 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 32666
I20260807 06:41:08.718777 885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:08.712909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40759 (local address 127.11.114.195:51327)
0807 06:41:08.713074 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:08.718704 (+ 5630us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260807 06:41:08.727427 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 436
I20260807 06:41:08.741819 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 626
2026-08-07T06:41:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18589 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:41:08 dist-test-slave-gkw7 krb5kdc[889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:41:08 dist-test-slave-gkw7 krb5kdc[889](info): set up 2 sockets
Aug 07 06:41:08 dist-test-slave-gkw7 krb5kdc[889](info): commencing operation
krb5kdc: starting...
W20260807 06:41:09.992344 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:41:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:41:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:41:10.120796 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39459
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:41:10.203646 904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:10.203925 904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:10.203969 904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:10.205143 904 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:41:10.205220 904 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:41:10.205242 904 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:41:10.205261 904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:41:10.205291 904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:10.205322 904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:10.205346 904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:41:10.205374 904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:41:10.206913 904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:10.207219 904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:10.207463 904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:10.213160 915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:10.213199 904 server_base.cc:1034] running on GCE node
W20260807 06:41:10.213191 912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:10.213178 911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:10.213833 904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:10.214128 904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:10.215332 904 hybrid_clock.cc:648] HybridClock initialized: now 1786084870215281 us; error 83 us; skew 500 ppm
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:10.218263 904 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:41:10.218750 904 webserver.cc:460] Webserver started at http://127.11.114.254:36439/ using document root <none> and password file <none>
I20260807 06:41:10.219007 904 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:10.219063 904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:10.219259 904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:41:10.220451 904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "99e461fd3a6044ee90fdb8929009c482"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ec170f808fe39940e78d6403e69688f5"
server_key_iv: "f8e86a6cea2a8f1825274a862ec6e317"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.220870 904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "99e461fd3a6044ee90fdb8929009c482"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ec170f808fe39940e78d6403e69688f5"
server_key_iv: "f8e86a6cea2a8f1825274a862ec6e317"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.222543 904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 06:41:10.223440 924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.223656 904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:41:10.223762 904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "99e461fd3a6044ee90fdb8929009c482"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ec170f808fe39940e78d6403e69688f5"
server_key_iv: "f8e86a6cea2a8f1825274a862ec6e317"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.223845 904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:10.248021 904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:10.250633 904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:10.250830 904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:10.255704 904 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39459
I20260807 06:41:10.255741 1000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39459 every 8 connection(s)
I20260807 06:41:10.256155 904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:41:10.256959 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 904
I20260807 06:41:10.257081 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:41:10.257305 11723 external_mini_cluster.cc:1428] Setting key c63d25aaa5c9b36acda74e29ccbca2df
I20260807 06:41:10.257300 1001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:41:10.261035 1001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Bootstrap starting.
I20260807 06:41:10.262008 1001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Neither blocks nor log segments found. Creating new log.
I20260807 06:41:10.262357 1001 log.cc:824] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:10.263247 1001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: No bootstrap required, opened a new log
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:41:10.266196 1001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:10.266350 1001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:41:10.266376 1001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e461fd3a6044ee90fdb8929009c482, State: Initialized, Role: FOLLOWER
I20260807 06:41:10.266552 1001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:10.266637 1001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:41:10.266662 1001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:41:10.266736 1001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:41:10.267504 1001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:10.267612 1001 leader_election.cc:302] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99e461fd3a6044ee90fdb8929009c482; no voters:
I20260807 06:41:10.267809 1001 leader_election.cc:288] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:41:10.267845 1008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:41:10.267977 1008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 1 LEADER]: Becoming Leader. State: Replica: 99e461fd3a6044ee90fdb8929009c482, State: Running, Role: LEADER
I20260807 06:41:10.268112 1001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:41:10.268159 1008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:10.268585 1008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99e461fd3a6044ee90fdb8929009c482. Latest consensus state: current_term: 1 leader_uuid: "99e461fd3a6044ee90fdb8929009c482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } } }
I20260807 06:41:10.268594 1011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99e461fd3a6044ee90fdb8929009c482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } } }
I20260807 06:41:10.268821 1011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: This master's current role is: LEADER
I20260807 06:41:10.269083 1008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: This master's current role is: LEADER
I20260807 06:41:10.269392 1023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:41:10.270185 1023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:41:10.270651 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.258034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43584 (local address 127.11.114.254:39459)
0807 06:41:10.258287 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:10.258296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:10.258324 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:41:10.259000 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:10.259008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:10.259061 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:10.259191 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:10.260189 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.261213 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.262541 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.262966 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.263068 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.266188 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:10.266192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:10.266192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:10.266203 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:10.268414 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.268867 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.268871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.268871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.268979 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.269243 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.269244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.269245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.269594 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:10.270223 (+ 629us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:10.270298 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 06:41:10.270447 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260807 06:41:10.272171 1023 catalog_manager.cc:1347] Generated new cluster ID: 33d5e728b01448159875dacb3a9f4b9b
I20260807 06:41:10.272229 1023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:41:10.300757 1023 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:41:10.301492 1023 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:41:10.307559 1023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Generated new TSK 0
I20260807 06:41:10.307772 1023 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:41:10.341300 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39459
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:41:10.446482 1041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:10.446738 1041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:10.446780 1041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:10.448024 1041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:10.448091 1041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:10.448135 1041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:41:10.449769 1041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:10.450049 1041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:10.450291 1041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:10.455420 1049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:10.455437 1051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:10.455595 1041 server_base.cc:1034] running on GCE node
W20260807 06:41:10.455425 1053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:10.455833 1041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:10.456079 1041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:10.457248 1041 hybrid_clock.cc:648] HybridClock initialized: now 1786084870457245 us; error 36 us; skew 500 ppm
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:10.459879 1041 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:41:10.460266 1041 webserver.cc:460] Webserver started at http://127.11.114.193:38851/ using document root <none> and password file <none>
I20260807 06:41:10.460516 1041 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:10.460561 1041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:10.460723 1041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:41:10.461875 1041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "32227af9b8a54e688eb2f7b5e48140b8"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ba62f41c93048cbd66deb31c57bca461"
server_key_iv: "938b651d34c7495e46c587a5620a1e9e"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.462241 1041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "32227af9b8a54e688eb2f7b5e48140b8"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ba62f41c93048cbd66deb31c57bca461"
server_key_iv: "938b651d34c7495e46c587a5620a1e9e"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.463739 1041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:10.464568 1062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.464783 1041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:10.464895 1041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "32227af9b8a54e688eb2f7b5e48140b8"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ba62f41c93048cbd66deb31c57bca461"
server_key_iv: "938b651d34c7495e46c587a5620a1e9e"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.464969 1041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:10.478549 1041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:10.480453 1041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:10.480618 1041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:10.480865 1041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:10.481199 1041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:41:10.481235 1041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.481257 1041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:41:10.481308 1041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.487303 1041 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:42655
I20260807 06:41:10.487406 1231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:42655 every 8 connection(s)
I20260807 06:41:10.487722 1041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:41:10.488049 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 1041
I20260807 06:41:10.488185 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:41:10.488397 11723 external_mini_cluster.cc:1428] Setting key 9048de36b92ea6974cf499367d968e4b
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
I20260807 06:41:10.499763 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.488624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:45747 (local address 127.11.114.254:39459)
0807 06:41:10.488691 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:10.488693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:10.488886 (+ 193us) server_negotiation.cc:407] Connection header received
0807 06:41:10.489670 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:10.489671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:10.489682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:10.489731 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:10.491089 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.492010 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.493396 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.493708 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.493773 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.496487 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:10.496490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:10.496490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:10.496501 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:10.498099 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.498619 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.498621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.498622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.498693 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.498986 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.498987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.498987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.499118 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:10.499210 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:10.499546 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 06:41:10.499659 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:41:10.499970 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.488509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:45747)
0807 06:41:10.488743 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:10.488792 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:10.489503 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:10.489826 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:10.489830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:10.490686 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:41:10.490976 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.490983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.492123 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.492124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.493291 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.493294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.493801 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.493803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.493970 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.494574 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:41:10.494584 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:41:10.496373 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:41:10.498221 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:10.498229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:10.498231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:10.498515 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:10.498771 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:10.498772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:10.498773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:10.498899 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:10.499199 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:41:10.499206 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:41:10.499394 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 06:41:10.499522 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 06:41:10.499703 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":119,"threads_started":1}
I20260807 06:41:10.500579 1232 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39459
I20260807 06:41:10.500686 1232 heartbeater.cc:464] Registering TS with master...
I20260807 06:41:10.500854 1232 heartbeater.cc:511] Master 127.11.114.254:39459 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
I20260807 06:41:10.501439 949 ts_manager.cc:194] Registered new tserver with Master: 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655)
I20260807 06:41:10.502189 949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:45747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:41:10.516835 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39459
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:41:10.605360 1242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:10.605600 1242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:10.605626 1242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:10.606717 1242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:10.606755 1242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:10.606781 1242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:41:10.608201 1242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:10.608384 1242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:10.608570 1242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:10.613974 1250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:10.613909 1255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:10.613935 1251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:10.614133 1242 server_base.cc:1034] running on GCE node
I20260807 06:41:10.614470 1242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:10.614691 1242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:10.615867 1242 hybrid_clock.cc:648] HybridClock initialized: now 1786084870615855 us; error 29 us; skew 500 ppm
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:10.618510 1242 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:41:10.618942 1242 webserver.cc:460] Webserver started at http://127.11.114.194:37447/ using document root <none> and password file <none>
I20260807 06:41:10.619185 1242 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:10.619256 1242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:10.619402 1242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:41:10.620544 1242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "907c9c43a3024dbf8caa8dad84140b6d"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "61b6bdf038c54255b3ca430cc20a34d9"
server_key_iv: "241c363cc8aec71f9807952c6e578b16"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.620918 1242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "907c9c43a3024dbf8caa8dad84140b6d"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "61b6bdf038c54255b3ca430cc20a34d9"
server_key_iv: "241c363cc8aec71f9807952c6e578b16"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.622458 1242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:41:10.623268 1266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.623459 1242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:10.623555 1242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "907c9c43a3024dbf8caa8dad84140b6d"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "61b6bdf038c54255b3ca430cc20a34d9"
server_key_iv: "241c363cc8aec71f9807952c6e578b16"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.623632 1242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:10.643069 1242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:10.645274 1242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:10.645468 1242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:10.646904 1242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:10.647282 1242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:41:10.647341 1242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.647385 1242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:41:10.647413 1242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.653723 1242 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:34119
I20260807 06:41:10.653769 1430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:34119 every 8 connection(s)
I20260807 06:41:10.654134 1242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:41:10.663132 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 1242
I20260807 06:41:10.663254 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:41:10.663465 11723 external_mini_cluster.cc:1428] Setting key 4b9c97da12ef687f99e06926e8201ef3
I20260807 06:41:10.664384 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.654973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:43493 (local address 127.11.114.254:39459)
0807 06:41:10.655045 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:10.655047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:10.655268 (+ 221us) server_negotiation.cc:407] Connection header received
0807 06:41:10.655736 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:10.655737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:10.655747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:10.655811 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:10.656810 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.657377 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.658555 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.658752 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.658810 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.661020 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:10.661022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:10.661023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:10.661029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:10.662625 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.663161 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.663162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.663163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.663293 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.663655 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.663657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.663657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.663790 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:10.663850 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:10.664211 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 06:41:10.664313 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:41:10.664688 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.654931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:43493)
0807 06:41:10.655117 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:10.655161 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:10.655651 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:10.655867 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:10.655871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:10.656484 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:41:10.656687 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.656693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.657474 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.657475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.658469 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.658472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.658830 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.658831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.658935 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.659447 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:41:10.659455 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:41:10.660927 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:41:10.662723 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:10.662730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:10.662732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:10.663054 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:10.663406 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:10.663407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:10.663408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:10.663570 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:10.663850 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:41:10.663860 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:41:10.664022 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 06:41:10.664170 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 06:41:10.664384 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260807 06:41:10.665264 1432 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39459
I20260807 06:41:10.665387 1432 heartbeater.cc:464] Registering TS with master...
I20260807 06:41:10.665639 1432 heartbeater.cc:511] Master 127.11.114.254:39459 requested a full tablet report, sending...
I20260807 06:41:10.666035 949 ts_manager.cc:194] Registered new tserver with Master: 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119)
I20260807 06:41:10.666447 949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:43493
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:41:10.692402 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39459
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:41:10.777693 1441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:10.777938 1441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:10.777978 1441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:10.779160 1441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:10.779230 1441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:10.779276 1441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:41:10.780778 1441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:10.781046 1441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:10.781292 1441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:10.786671 1447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:10.786669 1450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:10.786705 1441 server_base.cc:1034] running on GCE node
W20260807 06:41:10.786671 1448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:10.787061 1441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:10.787321 1441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:10.788488 1441 hybrid_clock.cc:648] HybridClock initialized: now 1786084870788468 us; error 39 us; skew 500 ppm
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:10.791291 1441 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:41:10.791880 1441 webserver.cc:460] Webserver started at http://127.11.114.195:44453/ using document root <none> and password file <none>
I20260807 06:41:10.792335 1441 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:10.792443 1441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:10.792625 1441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:41:10.793924 1441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "6d04d39836c746f4a3a891fb595ed5ea"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "cbfc97f94ddc0df38c0651e69c4b9daa"
server_key_iv: "2bfcd9d53ec2e9408be247c011bff7f7"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.794342 1441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "6d04d39836c746f4a3a891fb595ed5ea"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "cbfc97f94ddc0df38c0651e69c4b9daa"
server_key_iv: "2bfcd9d53ec2e9408be247c011bff7f7"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.795830 1441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:41:10.796682 1460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.796907 1441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:10.797026 1441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "6d04d39836c746f4a3a891fb595ed5ea"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "cbfc97f94ddc0df38c0651e69c4b9daa"
server_key_iv: "2bfcd9d53ec2e9408be247c011bff7f7"
server_key_version: "encryptionkey@0"
I20260807 06:41:10.797140 1441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:10.806206 1441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:10.808681 1441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:10.808856 1441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:10.809114 1441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:10.809592 1441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:41:10.809660 1441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.809706 1441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:41:10.809736 1441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:10.816294 1441 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:38407
I20260807 06:41:10.816336 1617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:38407 every 8 connection(s)
I20260807 06:41:10.816730 1441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:41:10.818434 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 1441
I20260807 06:41:10.818548 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:41:10.818739 11723 external_mini_cluster.cc:1428] Setting key e1d6bdd367f627d9a62c7bccb661b780
Aug 07 06:41:10 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:41:10.827792 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.817565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:50639 (local address 127.11.114.254:39459)
0807 06:41:10.817649 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:10.817652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:10.817881 (+ 229us) server_negotiation.cc:407] Connection header received
0807 06:41:10.818425 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:10.818426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:10.818439 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:10.818492 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:10.819754 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.820280 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.821307 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.821523 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.821643 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.824014 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:10.824015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:10.824016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:10.824025 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:10.825920 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.826532 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.826533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.826534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.826611 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.826984 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.826985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.826986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.827142 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:10.827214 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:10.827583 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 06:41:10.827675 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:41:10.828027 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.817510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:50639)
0807 06:41:10.817749 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:10.817787 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:10.818302 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:10.818582 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:10.818586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:10.819362 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:41:10.819659 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.819664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.820396 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.820397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.821213 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.821216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.821653 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.821654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.821803 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.822358 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:41:10.822366 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:41:10.823863 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:41:10.826009 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:10.826017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:10.826019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:10.826424 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:10.826695 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:10.826696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:10.826696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:10.826876 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:10.827242 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:41:10.827247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:41:10.827420 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 06:41:10.827543 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 06:41:10.827747 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260807 06:41:10.828510 1618 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39459
I20260807 06:41:10.828603 1618 heartbeater.cc:464] Registering TS with master...
I20260807 06:41:10.828815 1618 heartbeater.cc:511] Master 127.11.114.254:39459 requested a full tablet report, sending...
I20260807 06:41:10.829239 949 ts_manager.cc:194] Registered new tserver with Master: 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407)
I20260807 06:41:10.829720 949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:50639
I20260807 06:41:10.830394 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:41:10.837731 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.831772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43592 (local address 127.11.114.254:39459)
0807 06:41:10.831812 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:10.831813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:10.831941 (+ 128us) server_negotiation.cc:407] Connection header received
0807 06:41:10.831981 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:10.831982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:10.831990 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:10.832040 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:10.832557 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.833024 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.833841 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.834021 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.834074 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.835195 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:10.835196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:10.835196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:10.835219 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:10.836620 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.836963 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.836964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.836965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.837021 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:10.837208 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:10.837209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:10.837209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:10.837363 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:10.837439 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:10.837602 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 06:41:10.837667 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:41:10.839314 949 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:41:10.840925 949 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:41:10.848083 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.843828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33670 (local address 127.11.114.195:38407)
0807 06:41:10.844064 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:10.844068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:10.844077 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:41:10.844114 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:10.844117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:10.844132 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:10.844432 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:10.844496 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.845634 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.846843 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.847616 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.847702 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.847770 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:10.847937 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 06:41:10.847987 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":116,"threads_started":1}
I20260807 06:41:10.848083 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.843421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42655 (local address 127.0.0.1:58364)
0807 06:41:10.843584 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:10.843626 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:10.843698 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:10.844234 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:10.844237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:10.844240 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:10.844349 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.844352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.845750 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.845751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.846996 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.846999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.847785 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.847786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.847840 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.847842 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:41:10.847882 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 06:41:10.847944 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":60,"thread_start_us":62,"threads_started":1}
I20260807 06:41:10.848094 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.843427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38407 (local address 127.0.0.1:33670)
0807 06:41:10.843584 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:10.843625 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:10.843696 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:10.844234 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:10.844237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:10.844241 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:10.844349 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.844352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.845769 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.845770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.846746 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.846748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.847716 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.847718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.847811 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.847828 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 06:41:10.847910 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 06:41:10.847944 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260807 06:41:10.848250 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.843656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:34119 (local address 127.0.0.1:42272)
0807 06:41:10.843824 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:10.843834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:10.843880 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:10.844303 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:10.844304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:10.844307 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:10.844372 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.844374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.845375 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.845376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.846886 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:10.846890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.847836 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.847837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:10.847908 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.847914 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:41:10.847969 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 06:41:10.848011 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260807 06:41:10.848460 1639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.843795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.11.114.194:34119)
0807 06:41:10.844023 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:10.844028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:10.844042 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:41:10.844149 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:10.844152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:10.844170 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:10.844467 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:10.844532 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.845261 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.846983 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.847682 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.847745 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.847793 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:10.848332 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 06:41:10.848382 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":111,"threads_started":1}
I20260807 06:41:10.848618 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:10.843513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.11.114.193:42655)
0807 06:41:10.844010 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:10.844016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:10.844027 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:41:10.844075 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:10.844077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:10.844094 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:10.844422 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:10.844496 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.845642 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.847101 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:10.847654 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:10.848051 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:10.848118 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:10.848434 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 06:41:10.848486 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":93,"threads_started":1}
I20260807 06:41:10.849397 1528 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee17d8505034075bfd22793ae3b6a66 (DEFAULT_TABLE table=test-table [id=56431bdc078f4f6480779b8347e1e0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:41:10.849399 1136 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee17d8505034075bfd22793ae3b6a66 (DEFAULT_TABLE table=test-table [id=56431bdc078f4f6480779b8347e1e0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:41:10.849812 1136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee17d8505034075bfd22793ae3b6a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:41:10.849812 1528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee17d8505034075bfd22793ae3b6a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:41:10.850049 1344 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee17d8505034075bfd22793ae3b6a66 (DEFAULT_TABLE table=test-table [id=56431bdc078f4f6480779b8347e1e0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:41:10.850404 1344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee17d8505034075bfd22793ae3b6a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:41:10.853247 1642 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Bootstrap starting.
I20260807 06:41:10.853550 1644 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Bootstrap starting.
I20260807 06:41:10.853569 1645 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Bootstrap starting.
I20260807 06:41:10.854066 1642 tablet_bootstrap.cc:654] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Neither blocks nor log segments found. Creating new log.
I20260807 06:41:10.854238 1644 tablet_bootstrap.cc:654] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Neither blocks nor log segments found. Creating new log.
I20260807 06:41:10.854229 1645 tablet_bootstrap.cc:654] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Neither blocks nor log segments found. Creating new log.
I20260807 06:41:10.854676 1642 log.cc:824] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:10.854873 1644 log.cc:824] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:10.854873 1645 log.cc:824] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:10.855508 1642 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: No bootstrap required, opened a new log
I20260807 06:41:10.855607 1642 ts_tablet_manager.cc:1397] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:41:10.855736 1644 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: No bootstrap required, opened a new log
I20260807 06:41:10.855839 1644 ts_tablet_manager.cc:1397] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:41:10.855954 1645 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: No bootstrap required, opened a new log
I20260807 06:41:10.856033 1645 ts_tablet_manager.cc:1397] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:41:10.857700 1644 raft_consensus.cc:348] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:10.857698 1642 raft_consensus.cc:348] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:10.857884 1642 raft_consensus.cc:374] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:41:10.857885 1644 raft_consensus.cc:374] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:41:10.857932 1644 raft_consensus.cc:729] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32227af9b8a54e688eb2f7b5e48140b8, State: Initialized, Role: FOLLOWER
I20260807 06:41:10.857932 1642 raft_consensus.cc:729] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907c9c43a3024dbf8caa8dad84140b6d, State: Initialized, Role: FOLLOWER
I20260807 06:41:10.857970 1645 raft_consensus.cc:348] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:10.858052 1642 consensus_queue.cc:260] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:10.858078 1644 consensus_queue.cc:260] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:10.858122 1645 raft_consensus.cc:374] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:41:10.858175 1645 raft_consensus.cc:729] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d04d39836c746f4a3a891fb595ed5ea, State: Initialized, Role: FOLLOWER
I20260807 06:41:10.858408 1642 ts_tablet_manager.cc:1428] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 06:41:10.858445 1644 ts_tablet_manager.cc:1428] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:41:10.858538 1432 heartbeater.cc:503] Master 127.11.114.254:39459 was elected leader, sending a full tablet report...
I20260807 06:41:10.858579 1232 heartbeater.cc:503] Master 127.11.114.254:39459 was elected leader, sending a full tablet report...
I20260807 06:41:10.858460 1645 consensus_queue.cc:260] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:10.859047 1645 ts_tablet_manager.cc:1428] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:41:10.859181 1618 heartbeater.cc:503] Master 127.11.114.254:39459 was elected leader, sending a full tablet report...
W20260807 06:41:10.904726 1433 tablet.cc:2369] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:41:10.988534 1233 tablet.cc:2369] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:41:11.067358 1621 tablet.cc:2369] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:41:11.204519 1655 raft_consensus.cc:482] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:41:11.204668 1655 raft_consensus.cc:504] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:11.205029 1655 leader_election.cc:288] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655), 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119)
I20260807 06:41:11.207937 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:11.205224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:34119 (local address 127.11.114.195:54923)
0807 06:41:11.205292 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:11.205306 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:11.205358 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:11.205557 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:11.205558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:11.205559 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:11.205637 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:11.205638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.206314 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.206316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:11.207114 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:11.207116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.207674 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.207676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:11.207752 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:11.207755 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:41:11.207821 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:41:11.207863 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":44}
I20260807 06:41:11.207935 1639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:11.205353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:54923 (local address 127.11.114.194:34119)
0807 06:41:11.205403 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:11.205405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:11.205414 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:41:11.205444 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:11.205445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:11.205457 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:11.205532 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:11.205720 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.206224 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.207212 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.207573 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.207654 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:11.207665 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:11.207830 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 06:41:11.207866 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:41:11.208045 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:11.205264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42655 (local address 127.11.114.195:58917)
0807 06:41:11.205464 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:11.205474 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:11.205575 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:11.205731 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:11.205732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:11.205733 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:11.205813 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:11.205814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.206437 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.206438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:11.207211 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:11.207213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.207772 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.207774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:11.207853 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:11.207857 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:41:11.207919 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 06:41:11.207959 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":126,"threads_started":1}
I20260807 06:41:11.208036 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:11.205364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:58917 (local address 127.11.114.193:42655)
0807 06:41:11.205411 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:11.205413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:11.205552 (+ 139us) server_negotiation.cc:407] Connection header received
0807 06:41:11.205636 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:11.205637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:11.205648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:11.205728 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:11.205886 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.206352 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.207305 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.207685 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.207772 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:11.207780 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:11.207938 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 06:41:11.207975 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:41:11.208340 1372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee17d8505034075bfd22793ae3b6a66" candidate_uuid: "6d04d39836c746f4a3a891fb595ed5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907c9c43a3024dbf8caa8dad84140b6d" is_pre_election: true
I20260807 06:41:11.208349 1167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee17d8505034075bfd22793ae3b6a66" candidate_uuid: "6d04d39836c746f4a3a891fb595ed5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32227af9b8a54e688eb2f7b5e48140b8" is_pre_election: true
I20260807 06:41:11.208509 1372 raft_consensus.cc:2454] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d04d39836c746f4a3a891fb595ed5ea in term 0.
I20260807 06:41:11.208532 1167 raft_consensus.cc:2454] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d04d39836c746f4a3a891fb595ed5ea in term 0.
I20260807 06:41:11.208794 1467 leader_election.cc:302] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32227af9b8a54e688eb2f7b5e48140b8, 6d04d39836c746f4a3a891fb595ed5ea; no voters:
I20260807 06:41:11.208995 1655 raft_consensus.cc:2781] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:41:11.209074 1655 raft_consensus.cc:482] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:41:11.209110 1655 raft_consensus.cc:3037] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:41:11.209944 1655 raft_consensus.cc:504] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:11.210100 1655 leader_election.cc:288] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [CANDIDATE]: Term 1 election: Requested vote from peers 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655), 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119)
I20260807 06:41:11.210273 1372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee17d8505034075bfd22793ae3b6a66" candidate_uuid: "6d04d39836c746f4a3a891fb595ed5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907c9c43a3024dbf8caa8dad84140b6d"
I20260807 06:41:11.210281 1167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee17d8505034075bfd22793ae3b6a66" candidate_uuid: "6d04d39836c746f4a3a891fb595ed5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32227af9b8a54e688eb2f7b5e48140b8"
I20260807 06:41:11.210367 1372 raft_consensus.cc:3037] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:41:11.210363 1167 raft_consensus.cc:3037] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:41:11.211040 1372 raft_consensus.cc:2454] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d04d39836c746f4a3a891fb595ed5ea in term 1.
I20260807 06:41:11.211115 1167 raft_consensus.cc:2454] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d04d39836c746f4a3a891fb595ed5ea in term 1.
I20260807 06:41:11.211272 1463 leader_election.cc:302] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d04d39836c746f4a3a891fb595ed5ea, 907c9c43a3024dbf8caa8dad84140b6d; no voters:
I20260807 06:41:11.211386 1655 raft_consensus.cc:2781] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:41:11.211591 1655 raft_consensus.cc:686] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 LEADER]: Becoming Leader. State: Replica: 6d04d39836c746f4a3a891fb595ed5ea, State: Running, Role: LEADER
I20260807 06:41:11.211699 1655 consensus_queue.cc:237] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:11.212566 948 catalog_manager.cc:5495] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195). New cstate: current_term: 1 leader_uuid: "6d04d39836c746f4a3a891fb595ed5ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } health_report { overall_health: UNKNOWN } } }
I20260807 06:41:11.291635 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:11.289015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.11.114.195:38407)
0807 06:41:11.289117 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:11.289120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:11.289130 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:41:11.289204 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:11.289205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:11.289224 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:11.289304 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:41:11.289458 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.289979 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.290803 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:11.290996 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:11.291084 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:11.291271 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:41:11.291382 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:41:11.291479 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:11.291499 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 06:41:11.291518 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 06:41:11.293593 1167 raft_consensus.cc:1261] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Refusing update from remote peer 6d04d39836c746f4a3a891fb595ed5ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:41:11.293594 1372 raft_consensus.cc:1261] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 1 FOLLOWER]: Refusing update from remote peer 6d04d39836c746f4a3a891fb595ed5ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:41:11.293985 1655 consensus_queue.cc:1035] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:41:11.294139 1657 consensus_queue.cc:1035] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:41:11.296423 1662 mvcc.cc:205] Tried to move back new op lower bound from 7315803632815546368 to 7315803632483753984. Current Snapshot: MvccSnapshot[applied={T|T < 7315803632815546368}]
I20260807 06:41:11.296705 1664 mvcc.cc:205] Tried to move back new op lower bound from 7315803632815546368 to 7315803632483753984. Current Snapshot: MvccSnapshot[applied={T|T < 7315803632815546368}]
I20260807 06:41:11.298287 1665 mvcc.cc:205] Tried to move back new op lower bound from 7315803632815546368 to 7315803632483753984. Current Snapshot: MvccSnapshot[applied={T|T < 7315803632815546368 or (T in {7315803632815546368})}]
I20260807 06:41:11.298707 948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:41:11.298810 948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:41:11.299597 948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Sending DeleteTablet for 3 replicas of tablet 8ee17d8505034075bfd22793ae3b6a66
I20260807 06:41:11.299903 1528 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee17d8505034075bfd22793ae3b6a66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:41:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:33670
I20260807 06:41:11.299954 1136 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee17d8505034075bfd22793ae3b6a66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:41:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:58364
I20260807 06:41:11.299984 1344 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee17d8505034075bfd22793ae3b6a66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:41:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:42272
I20260807 06:41:11.300107 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 1041
I20260807 06:41:11.300194 1676 tablet_replica.cc:330] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: stopping tablet replica
I20260807 06:41:11.300285 1676 raft_consensus.cc:2229] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:41:11.300390 1676 raft_consensus.cc:2258] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:41:11.310006 1676 ts_tablet_manager.cc:1905] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:41:11.312052 1676 ts_tablet_manager.cc:1918] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:41:11.312168 1676 log.cc:1196] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/8ee17d8505034075bfd22793ae3b6a66
I20260807 06:41:11.312414 1676 ts_tablet_manager.cc:1939] T 8ee17d8505034075bfd22793ae3b6a66 P 907c9c43a3024dbf8caa8dad84140b6d: Deleting consensus metadata
I20260807 06:41:11.313386 925 catalog_manager.cc:4841] TS 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119): tablet 8ee17d8505034075bfd22793ae3b6a66 (table test-table [id=56431bdc078f4f6480779b8347e1e0e7]) successfully deleted
I20260807 06:41:11.314733 1674 tablet_replica.cc:330] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: stopping tablet replica
I20260807 06:41:11.314903 1674 raft_consensus.cc:2229] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:41:11.315042 1674 raft_consensus.cc:2258] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:41:11.315524 1674 ts_tablet_manager.cc:1905] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:41:11.317567 1674 ts_tablet_manager.cc:1918] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:41:11.317656 1674 log.cc:1196] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/8ee17d8505034075bfd22793ae3b6a66
I20260807 06:41:11.317992 1674 ts_tablet_manager.cc:1939] T 8ee17d8505034075bfd22793ae3b6a66 P 6d04d39836c746f4a3a891fb595ed5ea: Deleting consensus metadata
I20260807 06:41:11.318521 925 catalog_manager.cc:4841] TS 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407): tablet 8ee17d8505034075bfd22793ae3b6a66 (table test-table [id=56431bdc078f4f6480779b8347e1e0e7]) successfully deleted
W20260807 06:41:11.319907 933 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:42655 (error 108)
I20260807 06:41:11.320377 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 1242
I20260807 06:41:11.320509 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:11.320358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42655 (local address 127.0.0.1:58374)
0807 06:41:11.320424 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:11.320472 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:42655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 06:41:11.325096 933 catalog_manager.cc:4584] TS 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ee17d8505034075bfd22793ae3b6a66: Network error: Client connection negotiation failed: client connection to 127.11.114.193:42655: connect: Connection refused (error 111)
I20260807 06:41:11.331312 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 1441
I20260807 06:41:11.341897 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 904
I20260807 06:41:31.353852 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39459
--webserver_interface=127.11.114.254
--webserver_port=36439
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:41:31.442065 1678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:31.442282 1678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:31.442304 1678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:31.443480 1678 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:41:31.443522 1678 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:41:31.443536 1678 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:41:31.443548 1678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:41:31.443614 1678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:31.443632 1678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:31.443652 1678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:41:31.443676 1678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:41:31.445194 1678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=36439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:31.445459 1678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:31.445767 1678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:31.454948 1690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:31.455005 1689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:31.455123 1678 server_base.cc:1034] running on GCE node
W20260807 06:41:31.455005 1692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:31.455474 1678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:31.455806 1678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:31.456986 1678 hybrid_clock.cc:648] HybridClock initialized: now 1786084891456968 us; error 35 us; skew 500 ppm
Aug 07 06:41:31 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084891, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:31.463644 1678 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:41:31.464073 1678 webserver.cc:460] Webserver started at http://127.11.114.254:36439/ using document root <none> and password file <none>
I20260807 06:41:31.464278 1678 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:31.464334 1678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:31.465770 1678 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:31.466490 1700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:31.466753 1678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:31.466856 1678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "99e461fd3a6044ee90fdb8929009c482"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ec170f808fe39940e78d6403e69688f5"
server_key_iv: "f8e86a6cea2a8f1825274a862ec6e317"
server_key_version: "encryptionkey@0"
I20260807 06:41:31.467289 1678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:31.485251 1678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:31.487483 1678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:31.487617 1678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:31.492033 1678 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39459
I20260807 06:41:31.492113 1771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39459 every 8 connection(s)
I20260807 06:41:31.492554 1678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:41:31.494249 1772 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:41:31.494930 1772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Bootstrap starting.
I20260807 06:41:31.500564 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 1678
I20260807 06:41:31.500906 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:42655
--local_ip_for_outbound_sockets=127.11.114.193
--tserver_master_addrs=127.11.114.254:39459
--webserver_port=38851
--webserver_interface=127.11.114.193
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:41:31.503449 1772 log.cc:824] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:31.505571 1772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:41:31.505893 1772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Bootstrap complete.
I20260807 06:41:31.508169 1772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:31.508435 1772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e461fd3a6044ee90fdb8929009c482, State: Initialized, Role: FOLLOWER
I20260807 06:41:31.508641 1772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:31.508737 1772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:41:31.508795 1772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:41:31.508844 1772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:41:31.509933 1772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:31.510052 1772 leader_election.cc:302] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99e461fd3a6044ee90fdb8929009c482; no voters:
I20260807 06:41:31.510232 1772 leader_election.cc:288] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:41:31.510314 1777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:41:31.510509 1772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:41:31.510571 1777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 2 LEADER]: Becoming Leader. State: Replica: 99e461fd3a6044ee90fdb8929009c482, State: Running, Role: LEADER
I20260807 06:41:31.510721 1777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:31.511045 1778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "99e461fd3a6044ee90fdb8929009c482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } } }
I20260807 06:41:31.511137 1778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: This master's current role is: LEADER
I20260807 06:41:31.511307 1778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99e461fd3a6044ee90fdb8929009c482. Latest consensus state: current_term: 2 leader_uuid: "99e461fd3a6044ee90fdb8929009c482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } } }
I20260807 06:41:31.511371 1778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: This master's current role is: LEADER
I20260807 06:41:31.511370 1791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:41:31.512310 1791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:41:31.512501 1791 catalog_manager.cc:1259] Loaded cluster ID: 33d5e728b01448159875dacb3a9f4b9b
I20260807 06:41:31.512760 1791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:41:31.513666 1791 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:41:31.513857 1791 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Loaded TSK: 0
I20260807 06:41:31.523648 1791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Generated new TSK 1
I20260807 06:41:31.523738 1791 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:41:31.599205 1774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:31.599440 1774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:31.599480 1774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:31.600634 1774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:31.600703 1774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:31.600754 1774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:41:31.602293 1774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:42655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=38851
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:31.602550 1774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:31.602792 1774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:31.607997 1813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:31.608012 1811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:31.608009 1816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:31.608642 1774 server_base.cc:1034] running on GCE node
I20260807 06:41:31.608865 1774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:31.609164 1774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:31.610338 1774 hybrid_clock.cc:648] HybridClock initialized: now 1786084891610319 us; error 33 us; skew 500 ppm
Aug 07 06:41:31 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084891, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:31.617105 1774 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:41:31.617664 1774 webserver.cc:460] Webserver started at http://127.11.114.193:38851/ using document root <none> and password file <none>
I20260807 06:41:31.617910 1774 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:31.618000 1774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:31.619454 1774 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:41:31.620196 1825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:31.620405 1774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:41:31.620509 1774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "32227af9b8a54e688eb2f7b5e48140b8"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ba62f41c93048cbd66deb31c57bca461"
server_key_iv: "938b651d34c7495e46c587a5620a1e9e"
server_key_version: "encryptionkey@0"
I20260807 06:41:31.620867 1774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:31.628165 1774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:31.630568 1774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:31.630760 1774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:31.630995 1774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:31.631464 1834 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:41:31.633440 1774 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:41:31.633502 1774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 06:41:31.633564 1774 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:41:31.634188 1774 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:41:31.634267 1774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 06:41:31.634327 1834 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Bootstrap starting.
I20260807 06:41:31.641898 1774 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:42655
I20260807 06:41:31.642299 1774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:41:31.645620 1834 log.cc:824] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:31.646569 1989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:42655 every 8 connection(s)
I20260807 06:41:31.647655 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 1774
I20260807 06:41:31.647972 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:34119
--local_ip_for_outbound_sockets=127.11.114.194
--tserver_master_addrs=127.11.114.254:39459
--webserver_port=37447
--webserver_interface=127.11.114.194
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:41:31.648439 1834 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:41:31.648759 1834 tablet_bootstrap.cc:492] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Bootstrap complete.
I20260807 06:41:31.650130 1834 ts_tablet_manager.cc:1397] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20260807 06:41:31.651580 1834 raft_consensus.cc:348] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:31.652002 1834 raft_consensus.cc:729] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32227af9b8a54e688eb2f7b5e48140b8, State: Initialized, Role: FOLLOWER
I20260807 06:41:31.652240 1834 consensus_queue.cc:260] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } } peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } }
I20260807 06:41:31.652590 1834 ts_tablet_manager.cc:1428] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Aug 07 06:41:31 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084891, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:41:31.659785 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.643187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:55365)
0807 06:41:31.645679 (+ 2492us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:31.645749 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:31.646379 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:31.647268 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:31.647272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:31.647698 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:41:31.648032 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.648039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.649859 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.649860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.650891 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.650894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.652010 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.652012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.652154 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.652797 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:41:31.652808 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:41:31.654834 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:41:31.657511 (+ 2677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:31.657519 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:31.657522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:31.657899 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:31.658238 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:31.658239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:31.658239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:31.658384 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:31.658928 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:41:31.658936 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:41:31.659152 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 06:41:31.659293 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 06:41:31.659537 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2442,"thread_start_us":102,"threads_started":1}
I20260807 06:41:31.659822 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.643290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:55365 (local address 127.11.114.254:39459)
0807 06:41:31.646875 (+ 3585us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.646884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.646919 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:41:31.647028 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.647032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.647074 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.647227 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:31.648190 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.649706 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.650994 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.651889 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.652498 (+ 609us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.654983 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:31.654988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:31.654989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:31.655004 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:31.657394 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.658000 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.658004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.658005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.658143 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.658487 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.658488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.658488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.658830 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:31.658932 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:31.659400 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 06:41:31.659580 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3536,"thread_start_us":89,"threads_started":1}
I20260807 06:41:31.660800 1992 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39459
I20260807 06:41:31.660917 1992 heartbeater.cc:464] Registering TS with master...
I20260807 06:41:31.661132 1992 heartbeater.cc:511] Master 127.11.114.254:39459 requested a full tablet report, sending...
I20260807 06:41:31.662029 1726 ts_manager.cc:194] Registered new tserver with Master: 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655)
I20260807 06:41:31.663094 1726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:55365
I20260807 06:41:31.664201 1992 heartbeater.cc:503] Master 127.11.114.254:39459 was elected leader, sending a full tablet report...
I20260807 06:41:31.672374 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.662651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42655 (local address 127.0.0.1:54920)
0807 06:41:31.668591 (+ 5940us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:31.668634 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:31.668746 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:31.669009 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:31.669013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:31.669015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:31.669093 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.669096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.670190 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.670191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.671242 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.671245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.672035 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.672037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.672125 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.672140 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:41:31.672227 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:41:31.672291 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5914,"thread_start_us":97,"threads_started":1}
I20260807 06:41:31.672374 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.662726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54920 (local address 127.11.114.193:42655)
0807 06:41:31.665668 (+ 2942us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.665674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.668726 (+ 3052us) server_negotiation.cc:407] Connection header received
0807 06:41:31.668865 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.668869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.668887 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.669001 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:31.669179 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.670079 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.671350 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.671935 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.672037 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.672097 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:31.672249 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 06:41:31.672292 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2896,"thread_start_us":98,"threads_started":1}
I20260807 06:41:31.672943 1886 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee17d8505034075bfd22793ae3b6a66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:41:11 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:54920
I20260807 06:41:31.673647 2005 tablet_replica.cc:330] stopping tablet replica
I20260807 06:41:31.673794 2005 raft_consensus.cc:2229] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:41:31.673887 2005 raft_consensus.cc:2258] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:41:31.674280 2005 ts_tablet_manager.cc:1905] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:41:31.676465 2005 ts_tablet_manager.cc:1918] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:41:31.676573 2005 log.cc:1196] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal/wals/8ee17d8505034075bfd22793ae3b6a66
I20260807 06:41:31.676843 2005 ts_tablet_manager.cc:1939] T 8ee17d8505034075bfd22793ae3b6a66 P 32227af9b8a54e688eb2f7b5e48140b8: Deleting consensus metadata
I20260807 06:41:31.677438 1704 catalog_manager.cc:4841] TS 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655): tablet 8ee17d8505034075bfd22793ae3b6a66 (table test-table [id=56431bdc078f4f6480779b8347e1e0e7]) successfully deleted
W20260807 06:41:31.741600 1999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:31.741843 1999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:31.741868 1999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:31.742966 1999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:31.743000 1999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:31.743027 1999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:41:31.744421 1999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:34119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=37447
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:31.744611 1999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:31.744797 1999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:31.749853 2015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:31.749859 2018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:31.749866 2013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:31.750346 1999 server_base.cc:1034] running on GCE node
I20260807 06:41:31.750597 1999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:31.750870 1999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:31.752045 1999 hybrid_clock.cc:648] HybridClock initialized: now 1786084891752020 us; error 54 us; skew 500 ppm
Aug 07 06:41:31 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084891, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:31.755631 1999 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:41:31.756333 1999 webserver.cc:460] Webserver started at http://127.11.114.194:37447/ using document root <none> and password file <none>
I20260807 06:41:31.756662 1999 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:31.756775 1999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:31.759050 1999 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:31.759827 2026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:31.760023 1999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:31.760179 1999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "907c9c43a3024dbf8caa8dad84140b6d"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "61b6bdf038c54255b3ca430cc20a34d9"
server_key_iv: "241c363cc8aec71f9807952c6e578b16"
server_key_version: "encryptionkey@0"
I20260807 06:41:31.760646 1999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:31.774921 1999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:31.777963 1999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:31.778152 1999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:31.778441 1999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:31.778842 1999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:41:31.778895 1999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:31.778942 1999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:41:31.778975 1999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:31.788358 1999 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:34119
I20260807 06:41:31.788400 2188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:34119 every 8 connection(s)
I20260807 06:41:31.788889 1999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:41:31.796695 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 1999
I20260807 06:41:31.797039 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:38407
--local_ip_for_outbound_sockets=127.11.114.195
--tserver_master_addrs=127.11.114.254:39459
--webserver_port=44453
--webserver_interface=127.11.114.195
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:41:31 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084891, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:41:31.801999 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.789745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:56485 (local address 127.11.114.254:39459)
0807 06:41:31.789805 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.789807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.790108 (+ 301us) server_negotiation.cc:407] Connection header received
0807 06:41:31.790773 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.790776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.790791 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.790847 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:31.792258 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.792825 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.794398 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.794653 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.794752 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.798191 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:31.798193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:31.798193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:31.798203 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:31.800008 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.800591 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.800592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.800593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.800661 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.800981 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.800982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.800982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.801164 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:31.801214 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:31.801676 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 06:41:31.801818 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:41:31.802876 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.789728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:56485)
0807 06:41:31.789939 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:31.789996 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:31.790637 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:31.790987 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:31.790992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:31.791778 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:41:31.792124 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.792130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.792934 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.792936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.794312 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.794315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.794763 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.794764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.794928 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.795578 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:41:31.795591 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:41:31.798046 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:41:31.800112 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:31.800120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:31.800123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:31.800484 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:31.800743 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:31.800744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:31.800744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:31.800896 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:31.801271 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:41:31.801278 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:41:31.801462 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 06:41:31.802309 (+ 847us) client_negotiation.cc:240] Negotiation successful
0807 06:41:31.802544 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260807 06:41:31.803622 2189 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39459
I20260807 06:41:31.803737 2189 heartbeater.cc:464] Registering TS with master...
I20260807 06:41:31.803969 2189 heartbeater.cc:511] Master 127.11.114.254:39459 requested a full tablet report, sending...
I20260807 06:41:31.804443 1726 ts_manager.cc:194] Registered new tserver with Master: 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119)
I20260807 06:41:31.804963 1726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:56485
W20260807 06:41:31.889127 2194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:31.889384 2194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:31.889422 2194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:31.890623 2194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:31.890697 2194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:31.890745 2194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:41:31.892220 2194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:38407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=44453
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:31.892488 2194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:31.892731 2194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:31.897927 2203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:31.898051 2204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:31.898051 2194 server_base.cc:1034] running on GCE node
W20260807 06:41:31.898187 2209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:31.898413 2194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:31.898672 2194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:31.899859 2194 hybrid_clock.cc:648] HybridClock initialized: now 1786084891899839 us; error 32 us; skew 500 ppm
Aug 07 06:41:31 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084891, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:31.902591 2194 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:41:31.903043 2194 webserver.cc:460] Webserver started at http://127.11.114.195:44453/ using document root <none> and password file <none>
I20260807 06:41:31.903292 2194 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:31.903381 2194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:31.905078 2194 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:41:31.905893 2216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:31.906055 2194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:31.906185 2194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "6d04d39836c746f4a3a891fb595ed5ea"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "cbfc97f94ddc0df38c0651e69c4b9daa"
server_key_iv: "2bfcd9d53ec2e9408be247c011bff7f7"
server_key_version: "encryptionkey@0"
I20260807 06:41:31.906639 2194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:31.939242 2194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:31.941915 2194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:31.942108 2194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:31.942389 2194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:31.942735 2194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:41:31.942792 2194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:31.942837 2194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:41:31.942865 2194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:31.948899 2194 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:38407
I20260807 06:41:31.948984 2372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:38407 every 8 connection(s)
I20260807 06:41:31.949366 2194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:41:31.954468 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 2194
Aug 07 06:41:31 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084891, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:41:31.960143 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.950171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:57193 (local address 127.11.114.254:39459)
0807 06:41:31.950259 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.950262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.950463 (+ 201us) server_negotiation.cc:407] Connection header received
0807 06:41:31.950939 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.950940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.950952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.951007 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:31.951987 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.952517 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.953719 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.953962 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.954050 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.956832 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:31.956833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:31.956834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:31.956841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:31.958545 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.959048 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.959049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.959049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.959102 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.959405 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.959405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.959406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.959565 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:31.959640 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:31.959921 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 06:41:31.960050 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:41:31.960299 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.950146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:57193)
0807 06:41:31.950338 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:31.950375 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:31.950854 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:31.951063 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:31.951066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:31.951672 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:41:31.951875 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.951880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.952638 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.952639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.953582 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.953585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.954105 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.954107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.954238 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.954901 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:41:31.954934 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:41:31.956739 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:41:31.958625 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:31.958633 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:31.958636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:31.958969 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:31.959191 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:31.959192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:31.959192 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:31.959338 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:31.959636 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:41:31.959641 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:41:31.959770 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 06:41:31.959879 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 06:41:31.960053 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
I20260807 06:41:31.960786 2373 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39459
I20260807 06:41:31.960908 2373 heartbeater.cc:464] Registering TS with master...
I20260807 06:41:31.961090 2373 heartbeater.cc:511] Master 127.11.114.254:39459 requested a full tablet report, sending...
I20260807 06:41:31.961592 1726 ts_manager.cc:194] Registered new tserver with Master: 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407)
I20260807 06:41:31.962106 1726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:57193
I20260807 06:41:31.962396 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.954970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46472 (local address 127.11.114.254:39459)
0807 06:41:31.955099 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.955101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.955204 (+ 103us) server_negotiation.cc:407] Connection header received
0807 06:41:31.955252 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.955253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.955268 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.955341 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:31.955956 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.956537 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.957627 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.957905 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.957989 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.959406 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:31.959408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:31.959408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:31.959414 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:31.961176 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.961571 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.961572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.961572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.961632 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.961890 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.961891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.961891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.961995 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:31.962120 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:31.962220 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 06:41:31.962304 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260807 06:41:31.963089 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:41:31.965778 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.963429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.11.114.254:39459)
0807 06:41:31.963466 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.963468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.963559 (+ 91us) server_negotiation.cc:407] Connection header received
0807 06:41:31.963610 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.963610 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.963644 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.963718 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:41:31.963834 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.964290 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.965049 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.965295 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.965347 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.965481 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:41:31.965525 (+ 44us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 06:41:31.965735 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46488: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 06:41:31.965821 2382 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46488: token expired
W20260807 06:41:31.965999 1624 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.114.254:39459)
I20260807 06:41:31.971778 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.966250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46498 (local address 127.11.114.254:39459)
0807 06:41:31.966284 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.966286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.966296 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:41:31.966362 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.966362 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.966369 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.966408 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:31.966859 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.967285 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.967919 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.968095 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.968156 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.969203 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:31.969204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:31.969204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:31.969210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:31.970738 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.971039 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.971040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.971040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.971091 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:31.971304 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:31.971304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:31.971305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:31.971472 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:31.971560 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:31.971656 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 06:41:31.971717 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 06:41:31.972980 1726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:41:31.974692 1726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:41:31.976919 1886 tablet_service.cc:1467] Processing CreateTablet for tablet a6859dd6efd748a0885fb5c6a7830053 (DEFAULT_TABLE table=test-table [id=dd0167fe85b7447bae2fee3ca25da5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:41:31.977178 1886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6859dd6efd748a0885fb5c6a7830053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:41:31.978762 1834 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Bootstrap starting.
I20260807 06:41:31.979166 1834 tablet_bootstrap.cc:654] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Neither blocks nor log segments found. Creating new log.
I20260807 06:41:31.979775 1834 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: No bootstrap required, opened a new log
I20260807 06:41:31.979822 1834 ts_tablet_manager.cc:1397] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:41:31.979933 1834 raft_consensus.cc:348] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:31.979997 1834 raft_consensus.cc:374] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:41:31.980028 1834 raft_consensus.cc:729] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32227af9b8a54e688eb2f7b5e48140b8, State: Initialized, Role: FOLLOWER
I20260807 06:41:31.980093 1834 consensus_queue.cc:260] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:31.980196 1834 ts_tablet_manager.cc:1428] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:41:31.980926 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.976741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38407 (local address 127.0.0.1:36156)
0807 06:41:31.976900 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:31.976909 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:31.976946 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:31.977396 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:31.977398 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:31.977401 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:31.977460 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.977462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.978544 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.978545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.979709 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.979712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.980666 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.980667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.980735 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.980742 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:41:31.980802 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:41:31.980845 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260807 06:41:31.981176 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.977007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.11.114.195:38407)
0807 06:41:31.977198 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.977204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.977214 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:41:31.977260 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.977261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.977293 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.977380 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:31.977576 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.978450 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.979823 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.980578 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.980656 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.980722 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:31.981040 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 06:41:31.981084 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":114,"threads_started":1}
I20260807 06:41:31.981959 2286 tablet_service.cc:1467] Processing CreateTablet for tablet a6859dd6efd748a0885fb5c6a7830053 (DEFAULT_TABLE table=test-table [id=dd0167fe85b7447bae2fee3ca25da5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:41:31.982327 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.976610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:34119 (local address 127.0.0.1:53464)
0807 06:41:31.976759 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:31.976773 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:31.976819 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:31.978994 (+ 2175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:31.978995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:31.978997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:31.979039 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.979041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.980276 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.980278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.981330 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:31.981333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.982063 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.982064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:31.982126 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.982130 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:41:31.982196 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:41:31.982245 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260807 06:41:31.982349 2384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:31.976711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.11.114.194:34119)
0807 06:41:31.978813 (+ 2102us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:31.978818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:31.978829 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:41:31.978871 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:31.978873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:31.978889 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:31.978985 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:31.979104 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.980157 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.981395 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:31.981977 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:31.982041 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:31.982074 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:31.982232 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 06:41:31.982277 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2059,"thread_start_us":130,"threads_started":1}
I20260807 06:41:31.982398 2286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6859dd6efd748a0885fb5c6a7830053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:41:31.983048 2097 tablet_service.cc:1467] Processing CreateTablet for tablet a6859dd6efd748a0885fb5c6a7830053 (DEFAULT_TABLE table=test-table [id=dd0167fe85b7447bae2fee3ca25da5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:41:31.983366 2097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6859dd6efd748a0885fb5c6a7830053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:41:31.985366 2389 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: Bootstrap starting.
I20260807 06:41:31.986127 2389 tablet_bootstrap.cc:654] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: Neither blocks nor log segments found. Creating new log.
I20260807 06:41:31.986441 2391 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Bootstrap starting.
I20260807 06:41:31.986735 2389 log.cc:824] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:31.987102 2391 tablet_bootstrap.cc:654] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Neither blocks nor log segments found. Creating new log.
I20260807 06:41:31.987457 2389 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: No bootstrap required, opened a new log
I20260807 06:41:31.987514 2389 ts_tablet_manager.cc:1397] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:41:31.987681 2391 log.cc:824] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:31.988396 2391 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: No bootstrap required, opened a new log
I20260807 06:41:31.988500 2391 ts_tablet_manager.cc:1397] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:41:31.988929 2389 raft_consensus.cc:348] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:31.989073 2389 raft_consensus.cc:374] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:41:31.989094 2389 raft_consensus.cc:729] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d04d39836c746f4a3a891fb595ed5ea, State: Initialized, Role: FOLLOWER
I20260807 06:41:31.989246 2389 consensus_queue.cc:260] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:31.989590 2389 ts_tablet_manager.cc:1428] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:41:31.989605 2373 heartbeater.cc:503] Master 127.11.114.254:39459 was elected leader, sending a full tablet report...
I20260807 06:41:31.990605 2391 raft_consensus.cc:348] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:31.990813 2391 raft_consensus.cc:374] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:41:31.990864 2391 raft_consensus.cc:729] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907c9c43a3024dbf8caa8dad84140b6d, State: Initialized, Role: FOLLOWER
I20260807 06:41:31.990984 2391 consensus_queue.cc:260] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:31.991256 2391 ts_tablet_manager.cc:1428] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:41:31.991294 2189 heartbeater.cc:503] Master 127.11.114.254:39459 was elected leader, sending a full tablet report...
W20260807 06:41:32.039651 2190 tablet.cc:2369] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:41:32.122152 2394 raft_consensus.cc:482] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:41:32.122354 2394 raft_consensus.cc:504] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:32.122761 2394 leader_election.cc:288] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119), 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655)
I20260807 06:41:32.126435 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:32.122994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42655 (local address 127.11.114.195:49241)
0807 06:41:32.123071 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:32.123084 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:32.123135 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:32.123318 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:32.123319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:32.123321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:32.123389 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:32.123390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.124043 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.124044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:32.125302 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:32.125305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.126156 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.126158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:32.126240 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:32.126244 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:41:32.126310 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:41:32.126364 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":13}
I20260807 06:41:32.126441 2396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:32.123014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:34119 (local address 127.11.114.195:36013)
0807 06:41:32.123235 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:32.123246 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:32.123338 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:32.123492 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:32.123492 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:32.123494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:41:32.123577 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:32.123579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.124319 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.124320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:32.125394 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:32.125396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.126178 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.126179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:32.126266 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:32.126269 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:41:32.126322 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 06:41:32.126351 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
I20260807 06:41:32.126484 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:32.123016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:49241 (local address 127.11.114.193:42655)
0807 06:41:32.123139 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:32.123141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:32.123179 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:41:32.123213 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:32.123214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:32.123225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:32.123298 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:32.123470 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.123960 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.125399 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.126053 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.126144 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:32.126156 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:32.126363 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 06:41:32.126412 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 06:41:32.126484 2384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:32.123110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:36013 (local address 127.11.114.194:34119)
0807 06:41:32.123215 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:32.123217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:32.123323 (+ 106us) server_negotiation.cc:407] Connection header received
0807 06:41:32.123401 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:32.123402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:32.123416 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:32.123480 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:41:32.123656 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.124215 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.125499 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.126107 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.126176 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:32.126183 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:32.126363 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 06:41:32.126412 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 06:41:32.126780 1920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6859dd6efd748a0885fb5c6a7830053" candidate_uuid: "6d04d39836c746f4a3a891fb595ed5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32227af9b8a54e688eb2f7b5e48140b8" is_pre_election: true
I20260807 06:41:32.126981 1920 raft_consensus.cc:2454] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d04d39836c746f4a3a891fb595ed5ea in term 0.
I20260807 06:41:32.126996 2126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6859dd6efd748a0885fb5c6a7830053" candidate_uuid: "6d04d39836c746f4a3a891fb595ed5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907c9c43a3024dbf8caa8dad84140b6d" is_pre_election: true
I20260807 06:41:32.127123 2126 raft_consensus.cc:2454] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d04d39836c746f4a3a891fb595ed5ea in term 0.
I20260807 06:41:32.127297 2223 leader_election.cc:302] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32227af9b8a54e688eb2f7b5e48140b8, 6d04d39836c746f4a3a891fb595ed5ea; no voters:
I20260807 06:41:32.127460 2394 raft_consensus.cc:2781] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:41:32.127532 2394 raft_consensus.cc:482] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:41:32.127563 2394 raft_consensus.cc:3037] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:41:32.128319 2394 raft_consensus.cc:504] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:32.128474 2394 leader_election.cc:288] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [CANDIDATE]: Term 1 election: Requested vote from peers 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119), 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655)
I20260807 06:41:32.128660 2126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6859dd6efd748a0885fb5c6a7830053" candidate_uuid: "6d04d39836c746f4a3a891fb595ed5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907c9c43a3024dbf8caa8dad84140b6d"
I20260807 06:41:32.128660 1920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6859dd6efd748a0885fb5c6a7830053" candidate_uuid: "6d04d39836c746f4a3a891fb595ed5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32227af9b8a54e688eb2f7b5e48140b8"
I20260807 06:41:32.128754 2126 raft_consensus.cc:3037] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:41:32.128754 1920 raft_consensus.cc:3037] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:41:32.129776 2126 raft_consensus.cc:2454] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d04d39836c746f4a3a891fb595ed5ea in term 1.
I20260807 06:41:32.129966 2220 leader_election.cc:302] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d04d39836c746f4a3a891fb595ed5ea, 907c9c43a3024dbf8caa8dad84140b6d; no voters:
I20260807 06:41:32.130082 2394 raft_consensus.cc:2781] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:41:32.130124 1920 raft_consensus.cc:2454] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d04d39836c746f4a3a891fb595ed5ea in term 1.
I20260807 06:41:32.130277 2394 raft_consensus.cc:686] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 LEADER]: Becoming Leader. State: Replica: 6d04d39836c746f4a3a891fb595ed5ea, State: Running, Role: LEADER
I20260807 06:41:32.130384 2394 consensus_queue.cc:237] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:32.131409 1726 catalog_manager.cc:5495] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195). New cstate: current_term: 1 leader_uuid: "6d04d39836c746f4a3a891fb595ed5ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } health_report { overall_health: UNKNOWN } } }
W20260807 06:41:32.147132 1993 tablet.cc:2369] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:41:32.168267 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:32.165706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.11.114.195:38407)
0807 06:41:32.165779 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:32.165781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:32.165790 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:41:32.165832 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:32.165833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:32.165860 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:32.165943 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:41:32.166078 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.166571 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.167342 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:32.167569 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:32.167649 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:32.167838 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:41:32.167975 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:41:32.168087 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:32.168113 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 06:41:32.168142 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:41:32.170228 1920 raft_consensus.cc:1261] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Refusing update from remote peer 6d04d39836c746f4a3a891fb595ed5ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:41:32.170295 2126 raft_consensus.cc:1261] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 1 FOLLOWER]: Refusing update from remote peer 6d04d39836c746f4a3a891fb595ed5ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:41:32.170675 2399 consensus_queue.cc:1035] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:41:32.170817 2394 consensus_queue.cc:1035] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:41:32.173008 2401 mvcc.cc:205] Tried to move back new op lower bound from 7315803718326460416 to 7315803718166769664. Current Snapshot: MvccSnapshot[applied={T|T < 7315803718326460416}]
I20260807 06:41:32.173177 2405 mvcc.cc:205] Tried to move back new op lower bound from 7315803718326460416 to 7315803718166769664. Current Snapshot: MvccSnapshot[applied={T|T < 7315803718326460416}]
I20260807 06:41:32.173337 2404 mvcc.cc:205] Tried to move back new op lower bound from 7315803718326460416 to 7315803718166769664. Current Snapshot: MvccSnapshot[applied={T|T < 7315803718326460416}]
I20260807 06:41:32.175864 1724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:41:32.175958 1724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:41:32.176707 1724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Sending DeleteTablet for 3 replicas of tablet a6859dd6efd748a0885fb5c6a7830053
I20260807 06:41:32.177047 1886 tablet_service.cc:1514] Processing DeleteTablet for tablet a6859dd6efd748a0885fb5c6a7830053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:41:32 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:54920
I20260807 06:41:32.177117 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 1774
I20260807 06:41:32.177124 2097 tablet_service.cc:1514] Processing DeleteTablet for tablet a6859dd6efd748a0885fb5c6a7830053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:41:32 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:53464
I20260807 06:41:32.177186 2005 tablet_replica.cc:330] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: stopping tablet replica
I20260807 06:41:32.177237 2005 raft_consensus.cc:2229] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:41:32.177286 2005 raft_consensus.cc:2258] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:41:32.177749 2286 tablet_service.cc:1514] Processing DeleteTablet for tablet a6859dd6efd748a0885fb5c6a7830053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:41:32 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:36156
I20260807 06:41:32.180931 2422 tablet_replica.cc:330] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: stopping tablet replica
I20260807 06:41:32.181092 2422 raft_consensus.cc:2229] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:41:32.181268 2422 raft_consensus.cc:2258] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:41:32.181468 2421 tablet_replica.cc:330] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: stopping tablet replica
I20260807 06:41:32.181591 2421 raft_consensus.cc:2229] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:41:32.181687 2421 raft_consensus.cc:2258] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:41:32.182832 2421 ts_tablet_manager.cc:1905] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:41:32.184190 2422 ts_tablet_manager.cc:1905] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:41:32.184620 2421 ts_tablet_manager.cc:1918] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:41:32.184749 2421 log.cc:1196] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/a6859dd6efd748a0885fb5c6a7830053
I20260807 06:41:32.185107 2421 ts_tablet_manager.cc:1939] T a6859dd6efd748a0885fb5c6a7830053 P 907c9c43a3024dbf8caa8dad84140b6d: Deleting consensus metadata
I20260807 06:41:32.185657 1701 catalog_manager.cc:4841] TS 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119): tablet a6859dd6efd748a0885fb5c6a7830053 (table test-table [id=dd0167fe85b7447bae2fee3ca25da5b7]) successfully deleted
I20260807 06:41:32.186090 2422 ts_tablet_manager.cc:1918] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:41:32.186188 2422 log.cc:1196] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/a6859dd6efd748a0885fb5c6a7830053
I20260807 06:41:32.186455 2422 ts_tablet_manager.cc:1939] T a6859dd6efd748a0885fb5c6a7830053 P 6d04d39836c746f4a3a891fb595ed5ea: Deleting consensus metadata
I20260807 06:41:32.186875 1701 catalog_manager.cc:4841] TS 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407): tablet a6859dd6efd748a0885fb5c6a7830053 (table test-table [id=dd0167fe85b7447bae2fee3ca25da5b7]) successfully deleted
W20260807 06:41:32.195156 1704 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:42655 (error 108)
I20260807 06:41:32.195600 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 1999
I20260807 06:41:32.195768 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:32.195601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:42655 (local address 127.0.0.1:54928)
0807 06:41:32.195677 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:32.195726 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:42655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 06:41:32.204100 1704 catalog_manager.cc:4584] TS 32227af9b8a54e688eb2f7b5e48140b8 (127.11.114.193:42655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6859dd6efd748a0885fb5c6a7830053: Network error: Client connection negotiation failed: client connection to 127.11.114.193:42655: connect: Connection refused (error 111)
I20260807 06:41:32.210469 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 2194
I20260807 06:41:32.221143 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 1678
I20260807 06:41:52.229848 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39459
--webserver_interface=127.11.114.254
--webserver_port=36439
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:41:52.313411 2425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:52.313642 2425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:52.313668 2425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:52.314873 2425 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:41:52.314918 2425 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:41:52.314931 2425 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:41:52.314944 2425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:41:52.314956 2425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:52.315027 2425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:52.315045 2425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:41:52.315068 2425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:41:52.316576 2425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=36439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:52.316859 2425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:52.317118 2425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:52.322585 2430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:52.322595 2433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:52.322599 2431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:52.322618 2425 server_base.cc:1034] running on GCE node
I20260807 06:41:52.323167 2425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:52.323499 2425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:52.324668 2425 hybrid_clock.cc:648] HybridClock initialized: now 1786084912324656 us; error 32 us; skew 500 ppm
Aug 07 06:41:52 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084912, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:52.327390 2425 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:41:52.327841 2425 webserver.cc:460] Webserver started at http://127.11.114.254:36439/ using document root <none> and password file <none>
I20260807 06:41:52.328100 2425 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:52.328177 2425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:52.329632 2425 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:52.330391 2442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:52.330628 2425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:52.330727 2425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "99e461fd3a6044ee90fdb8929009c482"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ec170f808fe39940e78d6403e69688f5"
server_key_iv: "f8e86a6cea2a8f1825274a862ec6e317"
server_key_version: "encryptionkey@0"
I20260807 06:41:52.331164 2425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:52.349954 2425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:52.352434 2425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:52.352612 2425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:52.356937 2425 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39459
I20260807 06:41:52.356992 2521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39459 every 8 connection(s)
I20260807 06:41:52.357407 2425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:41:52.359037 2522 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:41:52.359730 2522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Bootstrap starting.
I20260807 06:41:52.366588 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 2425
I20260807 06:41:52.366928 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:42655
--local_ip_for_outbound_sockets=127.11.114.193
--tserver_master_addrs=127.11.114.254:39459
--webserver_port=38851
--webserver_interface=127.11.114.193
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:41:52.368301 2522 log.cc:824] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:52.371636 2522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:41:52.372017 2522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Bootstrap complete.
I20260807 06:41:52.374507 2522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:52.374882 2522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e461fd3a6044ee90fdb8929009c482, State: Initialized, Role: FOLLOWER
I20260807 06:41:52.375128 2522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:52.375238 2522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:41:52.375294 2522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:41:52.375344 2522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:41:52.376340 2522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:52.376470 2522 leader_election.cc:302] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99e461fd3a6044ee90fdb8929009c482; no voters:
I20260807 06:41:52.376657 2522 leader_election.cc:288] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 06:41:52.376722 2527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:41:52.376850 2527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [term 3 LEADER]: Becoming Leader. State: Replica: 99e461fd3a6044ee90fdb8929009c482, State: Running, Role: LEADER
I20260807 06:41:52.376951 2522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:41:52.376938 2527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } }
I20260807 06:41:52.377207 2527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99e461fd3a6044ee90fdb8929009c482. Latest consensus state: current_term: 3 leader_uuid: "99e461fd3a6044ee90fdb8929009c482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } } }
I20260807 06:41:52.377287 2527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: This master's current role is: LEADER
I20260807 06:41:52.377213 2528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "99e461fd3a6044ee90fdb8929009c482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99e461fd3a6044ee90fdb8929009c482" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39459 } } }
I20260807 06:41:52.377347 2528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482 [sys.catalog]: This master's current role is: LEADER
I20260807 06:41:52.377480 2533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:41:52.378809 2533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:41:52.378942 2533 catalog_manager.cc:1259] Loaded cluster ID: 33d5e728b01448159875dacb3a9f4b9b
I20260807 06:41:52.378984 2533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:41:52.379856 2533 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:41:52.380062 2533 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Loaded TSK: 0
I20260807 06:41:52.380128 2533 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 99e461fd3a6044ee90fdb8929009c482: Loaded TSK: 1
F20260807 06:41:52.380354 2533 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcf4d1939ad google::LogMessage::Fail() at ??:0
@ 0x7fcf4d19688a google::LogMessage::SendToLog() at ??:0
@ 0x7fcf4d1934de google::LogMessage::Flush() at ??:0
@ 0x7fcf4d194a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcf4f900104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fcf4f8fdd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fcf550ed1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fcf550ed48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fcf550ed594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fcf550b3c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fcf550bdad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fcf4f0610d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fcf4f056954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcf4feba6db start_thread at ??:0
@ 0x7fcf4bc9161f clone at ??:0
W20260807 06:41:52.475101 2524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:52.475293 2524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:52.475317 2524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:52.476516 2524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:52.476558 2524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:52.476585 2524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:41:52.478086 2524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:42655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=38851
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:52.478538 2524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:52.478794 2524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:52.484097 2580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:52.484215 2584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:52.484330 2582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:52.484300 2524 server_base.cc:1034] running on GCE node
I20260807 06:41:52.484524 2524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:52.484843 2524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:52.486017 2524 hybrid_clock.cc:648] HybridClock initialized: now 1786084912486002 us; error 26 us; skew 500 ppm
Aug 07 06:41:52 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084912, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:52.488924 2524 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:41:52.489392 2524 webserver.cc:460] Webserver started at http://127.11.114.193:38851/ using document root <none> and password file <none>
I20260807 06:41:52.489722 2524 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:52.489804 2524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:52.491299 2524 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:52.492107 2596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:52.492316 2524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:52.492417 2524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "32227af9b8a54e688eb2f7b5e48140b8"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "ba62f41c93048cbd66deb31c57bca461"
server_key_iv: "938b651d34c7495e46c587a5620a1e9e"
server_key_version: "encryptionkey@0"
I20260807 06:41:52.492794 2524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:52.513346 2524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:52.515877 2524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:52.516067 2524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:52.516338 2524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:52.516803 2610 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:41:52.518833 2524 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:41:52.518899 2524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 06:41:52.518944 2524 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:41:52.519796 2524 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:41:52.519876 2524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 06:41:52.519939 2610 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Bootstrap starting.
I20260807 06:41:52.527576 2524 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:42655
I20260807 06:41:52.527767 2752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:42655 every 8 connection(s)
I20260807 06:41:52.528016 2524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:41:52.530819 2610 log.cc:824] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Log is configured to *not* fsync() on all Append() calls
I20260807 06:41:52.533591 2610 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:41:52.533946 2610 tablet_bootstrap.cc:492] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Bootstrap complete.
I20260807 06:41:52.533972 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 2524
I20260807 06:41:52.534258 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:34119
--local_ip_for_outbound_sockets=127.11.114.194
--tserver_master_addrs=127.11.114.254:39459
--webserver_port=37447
--webserver_interface=127.11.114.194
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:41:52.535128 2610 ts_tablet_manager.cc:1397] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.003s
I20260807 06:41:52.536484 2610 raft_consensus.cc:348] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:52.536794 2610 raft_consensus.cc:729] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32227af9b8a54e688eb2f7b5e48140b8, State: Initialized, Role: FOLLOWER
I20260807 06:41:52.537638 2610 consensus_queue.cc:260] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:52.537966 2610 ts_tablet_manager.cc:1428] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
W20260807 06:41:52.601621 2757 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:41:52.528669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:46811)
0807 06:41:52.528850 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.528893 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:52.529334 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:52.601343 (+ 72009us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
W20260807 06:41:52.601928 2753 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:39459 (0 consecutive failures): Network error: Failed to ping master at 127.11.114.254:39459: Client connection negotiation failed: client connection to 127.11.114.254:39459: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:41:52.602278 2757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:52.602146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:38819)
0807 06:41:52.602209 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.602250 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 06:41:52.602649 2757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:52.602559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:38893)
0807 06:41:52.602610 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.602635 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 06:41:52.602787 2753 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:41:52.639552 2759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:52.639825 2759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:52.639876 2759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:52.641136 2759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:52.641222 2759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:52.641273 2759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:41:52.642799 2759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:34119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=37447
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:52.643074 2759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:52.643337 2759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:52.648728 2769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:52.648772 2772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:52.648728 2770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:52.648978 2759 server_base.cc:1034] running on GCE node
I20260807 06:41:52.649210 2759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:52.649466 2759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:52.650633 2759 hybrid_clock.cc:648] HybridClock initialized: now 1786084912650609 us; error 44 us; skew 500 ppm
Aug 07 06:41:52 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084912, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:52.653128 2759 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:41:52.653607 2759 webserver.cc:460] Webserver started at http://127.11.114.194:37447/ using document root <none> and password file <none>
I20260807 06:41:52.653843 2759 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:52.653914 2759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:52.655287 2759 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:41:52.656044 2786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:52.656322 2759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:41:52.656442 2759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "907c9c43a3024dbf8caa8dad84140b6d"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "61b6bdf038c54255b3ca430cc20a34d9"
server_key_iv: "241c363cc8aec71f9807952c6e578b16"
server_key_version: "encryptionkey@0"
I20260807 06:41:52.656834 2759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:52.673087 2759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:52.675284 2759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:52.675462 2759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:52.675730 2759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:52.676074 2759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:41:52.676137 2759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:52.676173 2759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:41:52.676203 2759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:52.682641 2759 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:34119
I20260807 06:41:52.682770 2949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:34119 every 8 connection(s)
I20260807 06:41:52.683104 2759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:41:52.684422 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:52.683794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:43115)
0807 06:41:52.684169 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.684276 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":91,"threads_started":1}
W20260807 06:41:52.684825 2950 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:39459 (0 consecutive failures): Network error: Failed to ping master at 127.11.114.254:39459: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
I20260807 06:41:52.685125 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:52.685020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:37715)
0807 06:41:52.685074 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.685099 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 06:41:52.685379 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:52.685311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:39559)
0807 06:41:52.685338 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.685359 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260807 06:41:52.685489 2950 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:41:52.691078 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 2759
I20260807 06:41:52.691344 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:38407
--local_ip_for_outbound_sockets=127.11.114.195
--tserver_master_addrs=127.11.114.254:39459
--webserver_port=44453
--webserver_interface=127.11.114.195
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:41:52.773940 2953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:41:52.774307 2953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:41:52.774348 2953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:41:52.775787 2953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:41:52.775854 2953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:41:52.775900 2953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:41:52.777426 2953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:38407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=44453
--tserver_master_addrs=127.11.114.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:41:52.777706 2953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:41:52.777946 2953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:41:52.783241 2960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:52.783267 2961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:41:52.783358 2963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:41:52.783483 2953 server_base.cc:1034] running on GCE node
I20260807 06:41:52.783646 2953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:41:52.783840 2953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:41:52.784981 2953 hybrid_clock.cc:648] HybridClock initialized: now 1786084912784944 us; error 54 us; skew 500 ppm
Aug 07 06:41:52 dist-test-slave-gkw7 krb5kdc[889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084912, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:41:52.787653 2953 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:41:52.788131 2953 webserver.cc:460] Webserver started at http://127.11.114.195:44453/ using document root <none> and password file <none>
I20260807 06:41:52.788357 2953 fs_manager.cc:359] Metadata directory not provided
I20260807 06:41:52.788439 2953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:41:52.789852 2953 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:52.790518 2971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:52.790702 2953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:41:52.790817 2953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "6d04d39836c746f4a3a891fb595ed5ea"
format_stamp: "Formatted at 2026-08-07 06:41:10 on dist-test-slave-gkw7"
server_key: "cbfc97f94ddc0df38c0651e69c4b9daa"
server_key_iv: "2bfcd9d53ec2e9408be247c011bff7f7"
server_key_version: "encryptionkey@0"
I20260807 06:41:52.791143 2953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:41:52.803646 2953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:41:52.805742 2953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:41:52.805907 2953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:41:52.806129 2953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:41:52.806448 2953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:41:52.806509 2953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:52.806552 2953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:41:52.806578 2953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:41:52.812691 2953 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:38407
I20260807 06:41:52.812742 3134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:38407 every 8 connection(s)
I20260807 06:41:52.813084 2953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:41:52.814160 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:52.813796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:51803)
0807 06:41:52.813962 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.814028 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
W20260807 06:41:52.814558 3135 heartbeater.cc:644] Failed to heartbeat to 127.11.114.254:39459 (0 consecutive failures): Network error: Failed to ping master at 127.11.114.254:39459: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
I20260807 06:41:52.814847 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:52.814748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:55571)
0807 06:41:52.814800 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.814825 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:41:52.815131 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:52.815041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:46461)
0807 06:41:52.815088 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:52.815112 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:41:52.815248 3135 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:41:52.817082 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 2953
I20260807 06:41:53.603379 3141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:53.603129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:34873)
0807 06:41:53.603278 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:53.603315 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260807 06:41:53.686005 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:53.685781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:39725)
0807 06:41:53.685918 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:53.685950 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260807 06:41:53.815778 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:53.815584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:49757)
0807 06:41:53.815698 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:53.815729 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260807 06:41:54.284368 3146 raft_consensus.cc:482] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:41:54.284497 3146 raft_consensus.cc:504] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:54.284817 3146 leader_election.cc:288] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119), 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407)
Aug 07 06:41:54 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084912, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.195@KRBTEST.COM
Aug 07 06:41:54 dist-test-slave-gkw7 krb5kdc[889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084912, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.194@KRBTEST.COM
I20260807 06:41:54.295886 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:54.285338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38407 (local address 127.11.114.193:57433)
0807 06:41:54.285571 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:54.285583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:54.285644 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:54.286365 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:54.286366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:54.286779 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:41:54.286963 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:54.286969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:54.288055 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:54.288056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:54.288849 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:54.288852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:54.289474 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:54.289475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:54.289580 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:54.290266 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:41:54.290270 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:41:54.291850 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:41:54.294167 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:54.294173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:54.294176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:54.294395 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:54.294672 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:54.294672 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:54.294673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:54.294866 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:54.295315 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:41:54.295355 (+ 40us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:41:54.295479 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 06:41:54.295604 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 06:41:54.295783 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":62,"thread_start_us":49,"threads_started":1}
I20260807 06:41:54.295898 3154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:54.285452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:57433 (local address 127.11.114.195:38407)
0807 06:41:54.285641 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:54.285651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:54.285685 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:41:54.286183 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:54.286185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:54.286227 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:54.286382 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:54.287101 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:54.287936 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:54.288957 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:54.289378 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:54.289508 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:54.291960 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:54.291964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:54.291964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:54.291974 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:54.294064 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:54.294498 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:54.294501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:54.294501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:54.294591 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:54.294983 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:54.294984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:54.294985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:54.295217 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:54.295319 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:54.295658 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 06:41:54.295782 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":61,"threads_started":1}
I20260807 06:41:54.296271 3149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:54.285143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:58329 (local address 127.11.114.194:34119)
0807 06:41:54.285380 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 06:41:54.285393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:41:54.285435 (+ 42us) server_negotiation.cc:407] Connection header received
0807 06:41:54.286106 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:41:54.286108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:41:54.286154 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:41:54.286312 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:41:54.287096 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:54.287947 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:54.288943 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:41:54.289337 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:41:54.289471 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:54.292498 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:41:54.292502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:41:54.292502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:41:54.292513 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:41:54.294623 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:54.295031 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:54.295034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:54.295034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:54.295132 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:41:54.295358 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:41:54.295359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:41:54.295359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:41:54.295659 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:41:54.295754 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 06:41:54.296047 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 06:41:54.296190 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":93,"threads_started":1}
I20260807 06:41:54.296088 3148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:54.285033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:34119 (local address 127.11.114.193:58329)
0807 06:41:54.285291 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:54.285312 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:41:54.285387 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:41:54.286361 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:41:54.286364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:41:54.286780 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:41:54.286963 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:54.286968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:54.288060 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:54.288061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:54.288847 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:41:54.288850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:41:54.289446 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:41:54.289447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:41:54.289583 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:41:54.290189 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:41:54.290197 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:41:54.292376 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:41:54.294733 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:54.294734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:54.294734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:54.294928 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:54.295220 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:41:54.295221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:41:54.295221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:41:54.295285 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:41:54.295748 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:41:54.295749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:41:54.295798 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 06:41:54.295887 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:41:54.295992 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":5,"thread_start_us":174,"threads_started":1}
W20260807 06:41:54.297581 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
W20260807 06:41:54.297680 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
I20260807 06:41:54.297715 2597 leader_election.cc:302] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32227af9b8a54e688eb2f7b5e48140b8; no voters: 6d04d39836c746f4a3a891fb595ed5ea, 907c9c43a3024dbf8caa8dad84140b6d
I20260807 06:41:54.297809 3146 raft_consensus.cc:2726] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:41:54.603958 3148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:54.603831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:57117)
0807 06:41:54.603884 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:54.603920 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 06:41:54.686875 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:54.686555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:58831)
0807 06:41:54.686726 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:54.686814 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260807 06:41:54.816445 3156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:54.816245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:35705)
0807 06:41:54.816368 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:54.816407 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260807 06:41:55.604769 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:55.604515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:34979)
0807 06:41:55.604678 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:55.604719 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260807 06:41:55.687575 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:55.687334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:57135)
0807 06:41:55.687487 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:55.687531 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":61,"threads_started":1}
I20260807 06:41:55.817245 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:55.817019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:58015)
0807 06:41:55.817161 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:55.817204 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260807 06:41:56.294773 3163 raft_consensus.cc:482] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:41:56.294919 3163 raft_consensus.cc:504] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:56.295183 3163 leader_election.cc:288] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119), 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407)
W20260807 06:41:56.295676 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
W20260807 06:41:56.295754 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
I20260807 06:41:56.295773 2597 leader_election.cc:302] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32227af9b8a54e688eb2f7b5e48140b8; no voters: 6d04d39836c746f4a3a891fb595ed5ea, 907c9c43a3024dbf8caa8dad84140b6d
I20260807 06:41:56.295850 3163 raft_consensus.cc:2726] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:41:56.605518 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:56.605294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:43585)
0807 06:41:56.605430 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:56.605470 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260807 06:41:56.688323 3165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:56.688104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:36081)
0807 06:41:56.688224 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:56.688265 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260807 06:41:56.818095 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:56.817853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:48855)
0807 06:41:56.818000 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:56.818044 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260807 06:41:57.606407 3167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:57.606161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:46683)
0807 06:41:57.606317 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:57.606356 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260807 06:41:57.689010 3168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:57.688768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:57111)
0807 06:41:57.688918 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:57.688960 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260807 06:41:57.818810 3170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:57.818573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:45423)
0807 06:41:57.818726 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:57.818766 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260807 06:41:58.301343 3172 raft_consensus.cc:482] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:41:58.301463 3172 raft_consensus.cc:504] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:41:58.301709 3172 leader_election.cc:288] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119), 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407)
W20260807 06:41:58.302203 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
W20260807 06:41:58.302271 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
I20260807 06:41:58.302310 2597 leader_election.cc:302] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32227af9b8a54e688eb2f7b5e48140b8; no voters: 6d04d39836c746f4a3a891fb595ed5ea, 907c9c43a3024dbf8caa8dad84140b6d
I20260807 06:41:58.302389 3172 raft_consensus.cc:2726] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:41:58.607084 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:58.606851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:42815)
0807 06:41:58.607000 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:58.607037 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260807 06:41:58.689702 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:58.689477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:56147)
0807 06:41:58.689624 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:58.689658 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260807 06:41:58.819657 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:58.819293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:49367)
0807 06:41:58.819582 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:58.819618 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":61,"threads_started":1}
I20260807 06:41:59.607839 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:59.607582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:43351)
0807 06:41:59.607737 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:59.607790 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260807 06:41:59.690382 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:59.690141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:47851)
0807 06:41:59.690289 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:59.690334 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":61,"threads_started":1}
I20260807 06:41:59.820464 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:41:59.820222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:51685)
0807 06:41:59.820373 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:41:59.820417 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260807 06:42:00.171320 3185 raft_consensus.cc:482] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:42:00.171450 3185 raft_consensus.cc:504] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:42:00.171665 3185 leader_election.cc:288] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119), 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407)
W20260807 06:42:00.172078 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
W20260807 06:42:00.172163 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
I20260807 06:42:00.172183 2597 leader_election.cc:302] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32227af9b8a54e688eb2f7b5e48140b8; no voters: 6d04d39836c746f4a3a891fb595ed5ea, 907c9c43a3024dbf8caa8dad84140b6d
I20260807 06:42:00.172304 3185 raft_consensus.cc:2726] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:42:00.608551 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:00.608338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:42213)
0807 06:42:00.608470 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:00.608503 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260807 06:42:00.691129 3189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:00.690908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:37311)
0807 06:42:00.691040 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:00.691083 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260807 06:42:00.821131 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:00.820939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:40701)
0807 06:42:00.821058 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:00.821090 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260807 06:42:01.609334 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:01.609079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:48305)
0807 06:42:01.609235 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:01.609283 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260807 06:42:01.691861 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:01.691630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:49227)
0807 06:42:01.691771 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:01.691812 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260807 06:42:01.821954 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:01.821715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:46789)
0807 06:42:01.821865 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:01.821907 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260807 06:42:02.610039 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:02.609835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:56787)
0807 06:42:02.609959 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:02.609993 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260807 06:42:02.692601 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:02.692379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:53533)
0807 06:42:02.692513 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:02.692556 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260807 06:42:02.822670 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:02.822441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:49879)
0807 06:42:02.822587 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:02.822622 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260807 06:42:03.610802 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:03.610541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:54907)
0807 06:42:03.610700 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:03.610754 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260807 06:42:03.693708 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:03.693336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:51725)
0807 06:42:03.693612 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:03.693659 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":137,"threads_started":1}
I20260807 06:42:03.823443 3200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:03.823189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:45417)
0807 06:42:03.823357 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:03.823399 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260807 06:42:04.611562 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:04.611347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:41809)
0807 06:42:04.611476 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:04.611512 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260807 06:42:04.694398 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:04.694181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:37983)
0807 06:42:04.694316 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:04.694351 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260807 06:42:04.824258 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:04.823996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:49751)
0807 06:42:04.824163 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:04.824199 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260807 06:42:05.612380 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:05.612153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:43349)
0807 06:42:05.612289 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:05.612330 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260807 06:42:05.695139 3207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:05.694863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:49137)
0807 06:42:05.695060 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:05.695091 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":145,"threads_started":1}
I20260807 06:42:05.825094 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:05.824855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:55167)
0807 06:42:05.825006 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:05.825046 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260807 06:42:06.613055 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:06.612836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:45803)
0807 06:42:06.612972 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:06.613009 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260807 06:42:06.695847 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:06.695623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:60669)
0807 06:42:06.695763 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:06.695798 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260807 06:42:06.825861 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:06.825641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:40353)
0807 06:42:06.825775 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:06.825814 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260807 06:42:07.613940 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:07.613685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.193:47383)
0807 06:42:07.613854 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:07.613894 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260807 06:42:07.696574 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:07.696382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.194:33569)
0807 06:42:07.696497 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:07.696530 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260807 06:42:07.761615 3215 raft_consensus.cc:482] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:42:07.761710 3215 raft_consensus.cc:504] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "907c9c43a3024dbf8caa8dad84140b6d" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 34119 } } peers { permanent_uuid: "6d04d39836c746f4a3a891fb595ed5ea" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38407 } } peers { permanent_uuid: "32227af9b8a54e688eb2f7b5e48140b8" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 42655 } }
I20260807 06:42:07.761938 3215 leader_election.cc:288] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119), 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407)
W20260807 06:42:07.762393 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d04d39836c746f4a3a891fb595ed5ea (127.11.114.195:38407): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
W20260807 06:42:07.762512 2597 leader_election.cc:341] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 907c9c43a3024dbf8caa8dad84140b6d (127.11.114.194:34119): Not found: Tablet not found: a6859dd6efd748a0885fb5c6a7830053
I20260807 06:42:07.762531 2597 leader_election.cc:302] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32227af9b8a54e688eb2f7b5e48140b8; no voters: 6d04d39836c746f4a3a891fb595ed5ea, 907c9c43a3024dbf8caa8dad84140b6d
I20260807 06:42:07.762614 3215 raft_consensus.cc:2726] T a6859dd6efd748a0885fb5c6a7830053 P 32227af9b8a54e688eb2f7b5e48140b8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:42:07.821297 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 2524
I20260807 06:42:07.827526 3216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:07.827247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39459 (local address 127.11.114.195:34745)
0807 06:42:07.827429 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:07.827465 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.254:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":59,"threads_started":1}
I20260807 06:42:07.835418 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 2759
I20260807 06:42:07.841413 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 2953
2026-08-07T06:42:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59097 ms)
[----------] 30 tests from SecurityITest (243500 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:07 dist-test-slave-gkw7 krb5kdc[3223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:07 dist-test-slave-gkw7 krb5kdc[3223](info): set up 2 sockets
Aug 07 06:42:07 dist-test-slave-gkw7 krb5kdc[3223](info): commencing operation
krb5kdc: starting...
W20260807 06:42:09.093308 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:09.218532 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:34681
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:34681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:09.306530 3241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:09.306787 3241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:09.306829 3241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:09.307979 3241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:09.308056 3241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:09.308087 3241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:09.308125 3241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:09.308158 3241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:09.309777 3241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:34681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:34681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:09.310075 3241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:09.310330 3241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:09.316068 3246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:09.316062 3248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:09.316092 3241 server_base.cc:1034] running on GCE node
W20260807 06:42:09.316042 3251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:09.316452 3241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:09.316757 3241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:09.317920 3241 hybrid_clock.cc:648] HybridClock initialized: now 1786084929317896 us; error 64 us; skew 500 ppm
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:09.320571 3241 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:09.321015 3241 webserver.cc:460] Webserver started at http://127.11.114.254:35317/ using document root <none> and password file <none>
I20260807 06:42:09.321264 3241 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:09.321336 3241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:09.321485 3241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:09.322561 3241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "0cb7a257adff4225bfb5098b523b5772"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "27d8797ad5b3915bcc834a4601d51be4"
server_key_iv: "43da2bb2e87efc75d873915950afac62"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.322940 3241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "0cb7a257adff4225bfb5098b523b5772"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "27d8797ad5b3915bcc834a4601d51be4"
server_key_iv: "43da2bb2e87efc75d873915950afac62"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.324338 3241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:42:09.325219 3262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.325451 3241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:09.325587 3241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "0cb7a257adff4225bfb5098b523b5772"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "27d8797ad5b3915bcc834a4601d51be4"
server_key_iv: "43da2bb2e87efc75d873915950afac62"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.325670 3241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:09.338599 3241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:09.341269 3241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:09.341444 3241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:09.345422 3241 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:34681
I20260807 06:42:09.345453 3346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:34681 every 8 connection(s)
I20260807 06:42:09.345849 3241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:09.346758 3348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:09.349050 3348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772: Bootstrap starting.
I20260807 06:42:09.349749 3348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:09.350059 3348 log.cc:824] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:09.350759 3348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772: No bootstrap required, opened a new log
I20260807 06:42:09.352386 3348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb7a257adff4225bfb5098b523b5772" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 34681 } }
I20260807 06:42:09.352521 3348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:09.352545 3348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb7a257adff4225bfb5098b523b5772, State: Initialized, Role: FOLLOWER
I20260807 06:42:09.352653 3348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb7a257adff4225bfb5098b523b5772" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 34681 } }
I20260807 06:42:09.352743 3348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:09.352772 3348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:09.352818 3348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:09.353329 3348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb7a257adff4225bfb5098b523b5772" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 34681 } }
I20260807 06:42:09.353446 3348 leader_election.cc:302] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cb7a257adff4225bfb5098b523b5772; no voters:
I20260807 06:42:09.353652 3348 leader_election.cc:288] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:09.353696 3353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:09.353891 3353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [term 1 LEADER]: Becoming Leader. State: Replica: 0cb7a257adff4225bfb5098b523b5772, State: Running, Role: LEADER
I20260807 06:42:09.353946 3348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:09.353995 3353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb7a257adff4225bfb5098b523b5772" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 34681 } }
I20260807 06:42:09.354033 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 3241
I20260807 06:42:09.354131 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:09.354337 11723 external_mini_cluster.cc:1428] Setting key 0df25350ff99bb71e6a9606c2bff31ce
I20260807 06:42:09.354435 3353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cb7a257adff4225bfb5098b523b5772. Latest consensus state: current_term: 1 leader_uuid: "0cb7a257adff4225bfb5098b523b5772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb7a257adff4225bfb5098b523b5772" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 34681 } } }
I20260807 06:42:09.354504 3353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:09.354705 3354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cb7a257adff4225bfb5098b523b5772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb7a257adff4225bfb5098b523b5772" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 34681 } } }
I20260807 06:42:09.354794 3354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:09.355170 3363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:09.355890 3363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:09.357928 3363 catalog_manager.cc:1347] Generated new cluster ID: e1f7583281954d9f94c6764da45f3a3c
I20260807 06:42:09.357983 3363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:09.366387 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.354945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.11.114.254:34681)
0807 06:42:09.355133 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.355142 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.355175 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:42:09.355309 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.355317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.355376 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.355614 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:09.356506 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.357880 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.358994 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.359649 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.359766 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.362679 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:09.362683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:09.362683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:09.362696 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:09.364711 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.365130 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.365134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.365134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.365245 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.365461 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.365461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.365462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.365772 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:09.365916 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.366041 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.366168 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260807 06:42:09.374684 3363 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:09.375224 3363 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:09.380252 3363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772: Generated new TSK 0
I20260807 06:42:09.380450 3363 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:42:09.408221 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:34681
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:09.491093 3388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:09.491348 3388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:09.491387 3388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:09.492539 3388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:09.492616 3388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:09.492666 3388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:42:09.494181 3388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:34681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:09.494472 3388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:09.494709 3388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:09.499778 3397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:09.499886 3395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:09.499779 3394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:09.500159 3388 server_base.cc:1034] running on GCE node
I20260807 06:42:09.500362 3388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:09.500552 3388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:09.501684 3388 hybrid_clock.cc:648] HybridClock initialized: now 1786084929501675 us; error 28 us; skew 500 ppm
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:09.504226 3388 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:42:09.504648 3388 webserver.cc:460] Webserver started at http://127.11.114.193:38805/ using document root <none> and password file <none>
I20260807 06:42:09.504894 3388 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:09.504966 3388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:09.505120 3388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:09.506248 3388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "f29c5901336447f6b31889b46b402bae"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "182b118d0d39410f309143b99b297814"
server_key_iv: "7b020c95e18f96b767b04c145d5ab1fe"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.506613 3388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "f29c5901336447f6b31889b46b402bae"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "182b118d0d39410f309143b99b297814"
server_key_iv: "7b020c95e18f96b767b04c145d5ab1fe"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.507932 3388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:42:09.508679 3411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.508854 3388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:42:09.508960 3388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "f29c5901336447f6b31889b46b402bae"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "182b118d0d39410f309143b99b297814"
server_key_iv: "7b020c95e18f96b767b04c145d5ab1fe"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.509033 3388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:09.523025 3388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:09.524856 3388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:09.525022 3388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:09.525257 3388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:09.525593 3388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:09.525651 3388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.525687 3388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:09.525717 3388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.531842 3388 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:43739
I20260807 06:42:09.531919 3584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:43739 every 8 connection(s)
I20260807 06:42:09.532281 3388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:42:09.533571 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 3388
I20260807 06:42:09.533699 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:42:09.533941 11723 external_mini_cluster.cc:1428] Setting key 32013ba727136b251abb6993b103523e
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:09.543051 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.533033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:60509 (local address 127.11.114.254:34681)
0807 06:42:09.533123 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.533126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.533292 (+ 166us) server_negotiation.cc:407] Connection header received
0807 06:42:09.533822 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.533824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.533843 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.533925 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:09.535039 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.535806 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.537152 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.537355 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.537410 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.539883 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:09.539885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:09.539885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:09.539894 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:09.541614 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.542004 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.542006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.542006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.542077 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.542330 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.542331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.542332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.542494 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:09.542573 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.542835 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.542926 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:42:09.543069 3587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.532989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:34681 (local address 127.11.114.193:60509)
0807 06:42:09.533167 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.533203 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:09.533719 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:09.533973 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:09.533976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:09.534745 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:09.534939 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.534943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.535898 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.535899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.537049 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.537053 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.537444 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.537446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.537636 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.538136 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:09.538146 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:09.539803 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:09.541691 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:09.541698 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:09.541700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:09.541926 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:09.542153 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:09.542153 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:09.542154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:09.542260 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:09.542565 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:09.542570 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:09.542698 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 06:42:09.542781 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:42:09.542902 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260807 06:42:09.543668 3585 heartbeater.cc:347] Connected to a master server at 127.11.114.254:34681
I20260807 06:42:09.543797 3585 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:09.544042 3585 heartbeater.cc:511] Master 127.11.114.254:34681 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
I20260807 06:42:09.544611 3293 ts_manager.cc:194] Registered new tserver with Master: f29c5901336447f6b31889b46b402bae (127.11.114.193:43739)
I20260807 06:42:09.545315 3293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:60509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:42:09.562618 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:34681
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:09.645578 3591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:09.645805 3591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:09.645830 3591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:09.646921 3591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:09.646957 3591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:09.646983 3591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:42:09.648378 3591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:34681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:09.648568 3591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:09.648762 3591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:09.654096 3596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:09.654070 3602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:09.654070 3599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:09.654408 3591 server_base.cc:1034] running on GCE node
I20260807 06:42:09.654618 3591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:09.654811 3591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:09.655951 3591 hybrid_clock.cc:648] HybridClock initialized: now 1786084929655915 us; error 52 us; skew 500 ppm
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:09.658480 3591 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:42:09.658958 3591 webserver.cc:460] Webserver started at http://127.11.114.194:37589/ using document root <none> and password file <none>
I20260807 06:42:09.659186 3591 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:09.659262 3591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:09.659420 3591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:09.660454 3591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "4b9d6db2bff04d58a4501c86f7b16ae1"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "e8a94aa7709c32beca8deb9195aa7cac"
server_key_iv: "b3787c1477781368640b29b819f38d2c"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.660825 3591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "4b9d6db2bff04d58a4501c86f7b16ae1"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "e8a94aa7709c32beca8deb9195aa7cac"
server_key_iv: "b3787c1477781368640b29b819f38d2c"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.662312 3591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:09.663120 3609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.663337 3591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:09.663441 3591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "4b9d6db2bff04d58a4501c86f7b16ae1"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "e8a94aa7709c32beca8deb9195aa7cac"
server_key_iv: "b3787c1477781368640b29b819f38d2c"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.663522 3591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:09.679879 3591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:09.682181 3591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:09.682371 3591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:09.682627 3591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:09.682962 3591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:09.683020 3591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.683056 3591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:09.683104 3591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.689224 3591 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:39481
I20260807 06:42:09.689270 3770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:39481 every 8 connection(s)
I20260807 06:42:09.689635 3591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:09.698441 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 3591
I20260807 06:42:09.698575 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:42:09.698794 11723 external_mini_cluster.cc:1428] Setting key c283608d5ab61894e0a7c1bbbf805686
I20260807 06:42:09.699414 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.690513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:43333 (local address 127.11.114.254:34681)
0807 06:42:09.690586 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.690588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.690817 (+ 229us) server_negotiation.cc:407] Connection header received
0807 06:42:09.691331 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.691332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.691344 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.691399 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:09.692359 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.692868 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.693946 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.694133 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.694186 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.696361 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:09.696362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:09.696363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:09.696370 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:09.697788 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.698267 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.698268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.698269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.698323 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.698626 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.698627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.698628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.698772 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:09.698829 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.699231 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.699326 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:42:09.699600 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.690486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:34681 (local address 127.11.114.194:43333)
0807 06:42:09.690684 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.690723 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:09.691233 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:09.691458 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:09.691462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:09.692060 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:09.692270 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.692275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.692950 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.692950 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.693873 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.693876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.694207 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.694208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.694310 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.694806 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:09.694815 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:09.696257 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:09.697865 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:09.697871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:09.697873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:09.698116 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:09.698393 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:09.698394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:09.698395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:09.698559 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:09.698845 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:09.698853 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:09.699029 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 06:42:09.699198 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 06:42:09.699344 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260807 06:42:09.700182 3771 heartbeater.cc:347] Connected to a master server at 127.11.114.254:34681
I20260807 06:42:09.700269 3771 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:09.700474 3771 heartbeater.cc:511] Master 127.11.114.254:34681 requested a full tablet report, sending...
I20260807 06:42:09.700886 3293 ts_manager.cc:194] Registered new tserver with Master: 4b9d6db2bff04d58a4501c86f7b16ae1 (127.11.114.194:39481)
I20260807 06:42:09.701406 3293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:43333
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:42:09.726116 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:34681
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:09.809882 3779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:09.810132 3779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:09.810171 3779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:09.811491 3779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:09.811556 3779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:09.811601 3779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:42:09.813323 3779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:34681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:09.813629 3779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:09.813900 3779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:09.819088 3788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:09.819303 3791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:09.819324 3779 server_base.cc:1034] running on GCE node
W20260807 06:42:09.819128 3789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:09.819587 3779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:09.819839 3779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:09.820984 3779 hybrid_clock.cc:648] HybridClock initialized: now 1786084929820976 us; error 34 us; skew 500 ppm
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:09.823604 3779 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:42:09.824076 3779 webserver.cc:460] Webserver started at http://127.11.114.195:39001/ using document root <none> and password file <none>
I20260807 06:42:09.824339 3779 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:09.824410 3779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:09.824569 3779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:09.825686 3779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "4acfd745c5d04219a77068627659653a"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "27a216c0c1db0a256a51231ed743b375"
server_key_iv: "0d36874dabfe0dc5e19158adb92f5712"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.826053 3779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "4acfd745c5d04219a77068627659653a"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "27a216c0c1db0a256a51231ed743b375"
server_key_iv: "0d36874dabfe0dc5e19158adb92f5712"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.827431 3779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:09.828353 3805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.828572 3779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:09.828676 3779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "4acfd745c5d04219a77068627659653a"
format_stamp: "Formatted at 2026-08-07 06:42:09 on dist-test-slave-gkw7"
server_key: "27a216c0c1db0a256a51231ed743b375"
server_key_iv: "0d36874dabfe0dc5e19158adb92f5712"
server_key_version: "encryptionkey@0"
I20260807 06:42:09.828769 3779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:09.841856 3779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:09.843930 3779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:09.844092 3779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:09.844350 3779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:09.844678 3779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:09.844734 3779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.844769 3779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:09.844817 3779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:09.851274 3779 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:34043
I20260807 06:42:09.851326 3963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:34043 every 8 connection(s)
I20260807 06:42:09.851675 3779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:42:09.851987 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 3779
I20260807 06:42:09.852160 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:42:09.852389 11723 external_mini_cluster.cc:1428] Setting key 0d883ceaebf1200f407b0934fd69995f
Aug 07 06:42:09 dist-test-slave-gkw7 krb5kdc[3223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084929, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:09.861510 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.852473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:49645 (local address 127.11.114.254:34681)
0807 06:42:09.852527 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.852529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.852805 (+ 276us) server_negotiation.cc:407] Connection header received
0807 06:42:09.853324 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.853326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.853338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.853396 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:09.854529 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.855236 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.856335 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.856551 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.856638 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.858868 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:09.858871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:09.858871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:09.858879 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:09.860333 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.860686 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.860687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.860687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.860743 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.860952 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.860953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.860953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.861061 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:09.861143 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.861379 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.861444 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:42:09.861757 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.852481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:34681 (local address 127.11.114.195:49645)
0807 06:42:09.852643 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.852686 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:09.853211 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:09.853470 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:09.853474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:09.854167 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:09.854374 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.854379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.855340 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.855341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.856238 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.856241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.856650 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.856652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.856790 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.857260 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:09.857268 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:09.858769 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:09.860406 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:09.860412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:09.860413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:09.860619 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:09.860800 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:09.860801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:09.860801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:09.860898 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:09.861148 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:09.861152 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:09.861265 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 06:42:09.861348 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:42:09.861469 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260807 06:42:09.862319 3964 heartbeater.cc:347] Connected to a master server at 127.11.114.254:34681
I20260807 06:42:09.862406 3964 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:09.862608 3964 heartbeater.cc:511] Master 127.11.114.254:34681 requested a full tablet report, sending...
I20260807 06:42:09.863026 3293 ts_manager.cc:194] Registered new tserver with Master: 4acfd745c5d04219a77068627659653a (127.11.114.195:34043)
I20260807 06:42:09.863621 3293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:49645
I20260807 06:42:09.864629 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:42:09.871694 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.866024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35402 (local address 127.11.114.254:34681)
0807 06:42:09.866061 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.866063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.866179 (+ 116us) server_negotiation.cc:407] Connection header received
0807 06:42:09.866278 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.866279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.866288 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.866347 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:09.866861 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.867407 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.868090 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.868273 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.868324 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.869515 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:09.869516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:09.869516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:09.869522 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:09.870737 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.871060 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.871060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.871061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.871113 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:09.871279 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:09.871280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:09.871280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:09.871412 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:09.871486 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.871577 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.871633 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:42:09.873230 3293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:42:09.874902 3293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:42:09.881469 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.877253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34043 (local address 127.0.0.1:52442)
0807 06:42:09.877781 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.877788 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:09.877846 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:09.878174 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:09.878178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:09.878180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:09.878254 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.878257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.879165 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.879166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.880327 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.880330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.881138 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.881139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.881234 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.881249 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:42:09.881325 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:42:09.881379 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260807 06:42:09.881520 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.877474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.11.114.195:34043)
0807 06:42:09.877686 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.877691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.877842 (+ 151us) server_negotiation.cc:407] Connection header received
0807 06:42:09.877922 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.877925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.877943 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.878087 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:09.878327 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.879025 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.880413 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.881048 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.881128 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.881183 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.881356 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.881403 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":131,"threads_started":1}
I20260807 06:42:09.881850 3978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.877211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43739 (local address 127.0.0.1:58030)
0807 06:42:09.877556 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.877601 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:09.877672 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:09.878248 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:09.878249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:09.878250 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:09.878289 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.878290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.879616 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.879617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.880614 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.880616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.881530 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.881532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.881620 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.881626 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:42:09.881681 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 06:42:09.881726 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":61,"threads_started":1}
I20260807 06:42:09.882388 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.877316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.11.114.193:43739)
0807 06:42:09.877986 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.877990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.877999 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:09.878042 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.878044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.878145 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.878365 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:09.878452 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.879504 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.880712 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.881430 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.882110 (+ 680us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.882174 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.882264 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.882308 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":105,"threads_started":1}
I20260807 06:42:09.882676 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 86ce628e8fc44c459424687fda76ab21 (DEFAULT_TABLE table=test-table [id=a42aa66c9c64470a8474a54aad5eca64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:09.882988 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.877252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:39481 (local address 127.0.0.1:51770)
0807 06:42:09.877636 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.877650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:09.877708 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:09.878184 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:09.878185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:09.878187 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:09.878272 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.878275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.880709 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.880709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.881694 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.881698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.882757 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.882758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.882824 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.882828 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:09.882884 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:42:09.882923 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"spinlock_wait_cycles":896,"thread_start_us":65,"threads_started":1}
I20260807 06:42:09.883122 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86ce628e8fc44c459424687fda76ab21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:09.883225 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.877714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51770 (local address 127.11.114.194:39481)
0807 06:42:09.877915 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.877920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.877932 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:42:09.877974 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.877976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.877994 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.878077 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:09.878359 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.879497 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.881790 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.882646 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.882748 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.882801 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.883099 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.883140 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":127,"threads_started":1}
I20260807 06:42:09.883265 3490 tablet_service.cc:1467] Processing CreateTablet for tablet 86ce628e8fc44c459424687fda76ab21 (DEFAULT_TABLE table=test-table [id=a42aa66c9c64470a8474a54aad5eca64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:09.883616 3490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86ce628e8fc44c459424687fda76ab21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:09.886214 3984 tablet_bootstrap.cc:492] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: Bootstrap starting.
I20260807 06:42:09.886243 3678 tablet_service.cc:1467] Processing CreateTablet for tablet 86ce628e8fc44c459424687fda76ab21 (DEFAULT_TABLE table=test-table [id=a42aa66c9c64470a8474a54aad5eca64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:09.886535 3985 tablet_bootstrap.cc:492] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae: Bootstrap starting.
I20260807 06:42:09.886600 3678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86ce628e8fc44c459424687fda76ab21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:09.886942 3984 tablet_bootstrap.cc:654] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:09.887171 3985 tablet_bootstrap.cc:654] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:09.887620 3984 log.cc:824] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:09.887830 3985 log.cc:824] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:09.888470 3984 tablet_bootstrap.cc:492] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: No bootstrap required, opened a new log
I20260807 06:42:09.888516 3984 ts_tablet_manager.cc:1397] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:42:09.888612 3985 tablet_bootstrap.cc:492] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae: No bootstrap required, opened a new log
I20260807 06:42:09.888672 3985 ts_tablet_manager.cc:1397] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:42:09.889905 3988 tablet_bootstrap.cc:492] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Bootstrap starting.
I20260807 06:42:09.890141 3984 raft_consensus.cc:348] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.890345 3984 raft_consensus.cc:374] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:09.890381 3984 raft_consensus.cc:729] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acfd745c5d04219a77068627659653a, State: Initialized, Role: FOLLOWER
I20260807 06:42:09.890506 3984 consensus_queue.cc:260] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.890578 3988 tablet_bootstrap.cc:654] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:09.890653 3985 raft_consensus.cc:348] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.890820 3985 raft_consensus.cc:374] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:09.890861 3985 raft_consensus.cc:729] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29c5901336447f6b31889b46b402bae, State: Initialized, Role: FOLLOWER
I20260807 06:42:09.890856 3964 heartbeater.cc:503] Master 127.11.114.254:34681 was elected leader, sending a full tablet report...
I20260807 06:42:09.890990 3985 consensus_queue.cc:260] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.890838 3984 ts_tablet_manager.cc:1428] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:42:09.891281 3985 ts_tablet_manager.cc:1428] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:09.891348 3988 log.cc:824] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:09.891409 3585 heartbeater.cc:503] Master 127.11.114.254:34681 was elected leader, sending a full tablet report...
I20260807 06:42:09.892374 3988 tablet_bootstrap.cc:492] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: No bootstrap required, opened a new log
I20260807 06:42:09.892454 3988 ts_tablet_manager.cc:1397] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:42:09.893941 3988 raft_consensus.cc:348] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.894126 3988 raft_consensus.cc:374] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:09.894164 3988 raft_consensus.cc:729] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9d6db2bff04d58a4501c86f7b16ae1, State: Initialized, Role: FOLLOWER
I20260807 06:42:09.894256 3988 consensus_queue.cc:260] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.894456 3988 ts_tablet_manager.cc:1428] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:42:09.894527 3771 heartbeater.cc:503] Master 127.11.114.254:34681 was elected leader, sending a full tablet report...
I20260807 06:42:09.930585 3994 raft_consensus.cc:482] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:42:09.930735 3994 raft_consensus.cc:504] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.931149 3994 leader_election.cc:288] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f29c5901336447f6b31889b46b402bae (127.11.114.193:43739), 4acfd745c5d04219a77068627659653a (127.11.114.195:34043)
I20260807 06:42:09.934736 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.931399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34043 (local address 127.11.114.194:56769)
0807 06:42:09.931611 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.931622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:09.931695 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:09.931837 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:09.931838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:09.931839 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:09.931903 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.931904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.932526 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.932528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.933890 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.933893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.934462 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.934463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.934539 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.934543 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:09.934597 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 06:42:09.934629 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":79,"thread_start_us":132,"threads_started":1}
I20260807 06:42:09.934746 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.931483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:56769 (local address 127.11.114.195:34043)
0807 06:42:09.931570 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.931572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.931702 (+ 130us) server_negotiation.cc:407] Connection header received
0807 06:42:09.931750 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.931751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.931763 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.931814 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:09.931970 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.932439 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.933980 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.934375 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.934468 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.934479 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.934629 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.934676 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:42:09.934818 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.931402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:55933 (local address 127.11.114.193:43739)
0807 06:42:09.931450 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.931452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.931461 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:09.931489 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.931490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.931521 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.931576 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:09.931788 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.932422 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.934015 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.934385 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.934477 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.934485 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.934694 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.934737 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:42:09.934860 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.931301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43739 (local address 127.11.114.194:55933)
0807 06:42:09.931360 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.931372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:09.931415 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:09.931612 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:09.931614 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:09.931616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:09.931712 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.931714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.932601 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.932602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.933890 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:09.933893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.934500 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.934501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:09.934603 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.934608 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:09.934670 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 06:42:09.934709 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":54}
I20260807 06:42:09.935314 3523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86ce628e8fc44c459424687fda76ab21" candidate_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f29c5901336447f6b31889b46b402bae" is_pre_election: true
I20260807 06:42:09.935359 3900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86ce628e8fc44c459424687fda76ab21" candidate_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acfd745c5d04219a77068627659653a" is_pre_election: true
I20260807 06:42:09.935480 3523 raft_consensus.cc:2454] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b9d6db2bff04d58a4501c86f7b16ae1 in term 0.
I20260807 06:42:09.935508 3900 raft_consensus.cc:2454] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b9d6db2bff04d58a4501c86f7b16ae1 in term 0.
I20260807 06:42:09.935721 3612 leader_election.cc:302] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b9d6db2bff04d58a4501c86f7b16ae1, f29c5901336447f6b31889b46b402bae; no voters:
I20260807 06:42:09.935918 3994 raft_consensus.cc:2781] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:42:09.935997 3994 raft_consensus.cc:482] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:42:09.936033 3994 raft_consensus.cc:3037] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:09.936767 3994 raft_consensus.cc:504] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.936934 3994 leader_election.cc:288] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [CANDIDATE]: Term 1 election: Requested vote from peers f29c5901336447f6b31889b46b402bae (127.11.114.193:43739), 4acfd745c5d04219a77068627659653a (127.11.114.195:34043)
I20260807 06:42:09.937083 3900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86ce628e8fc44c459424687fda76ab21" candidate_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acfd745c5d04219a77068627659653a"
I20260807 06:42:09.937107 3523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86ce628e8fc44c459424687fda76ab21" candidate_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f29c5901336447f6b31889b46b402bae"
I20260807 06:42:09.937160 3900 raft_consensus.cc:3037] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:09.937196 3523 raft_consensus.cc:3037] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:09.937937 3900 raft_consensus.cc:2454] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b9d6db2bff04d58a4501c86f7b16ae1 in term 1.
I20260807 06:42:09.938052 3523 raft_consensus.cc:2454] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b9d6db2bff04d58a4501c86f7b16ae1 in term 1.
I20260807 06:42:09.938125 3615 leader_election.cc:302] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acfd745c5d04219a77068627659653a, 4b9d6db2bff04d58a4501c86f7b16ae1; no voters:
I20260807 06:42:09.938223 3994 raft_consensus.cc:2781] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:09.938410 3994 raft_consensus.cc:686] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 1 LEADER]: Becoming Leader. State: Replica: 4b9d6db2bff04d58a4501c86f7b16ae1, State: Running, Role: LEADER
I20260807 06:42:09.938530 3994 consensus_queue.cc:237] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } }
I20260807 06:42:09.939409 3286 catalog_manager.cc:5495] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b9d6db2bff04d58a4501c86f7b16ae1 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b9d6db2bff04d58a4501c86f7b16ae1" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 39481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 } health_report { overall_health: UNKNOWN } } }
W20260807 06:42:09.940205 3773 tablet.cc:2369] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:42:09.955637 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.952950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51778 (local address 127.11.114.194:39481)
0807 06:42:09.953013 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:09.953015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:09.953024 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:09.953074 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:09.953075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:09.953092 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:09.953153 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:09.953277 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.953952 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.954739 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:09.955044 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:09.955113 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:09.955290 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:09.955391 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:09.955463 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:09.955482 (+ 19us) server_negotiation.cc:299] Negotiation successful
0807 06:42:09.955500 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:42:09.957427 3523 raft_consensus.cc:1261] T 86ce628e8fc44c459424687fda76ab21 P f29c5901336447f6b31889b46b402bae [term 1 FOLLOWER]: Refusing update from remote peer 4b9d6db2bff04d58a4501c86f7b16ae1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:09.957451 3900 raft_consensus.cc:1261] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 1 FOLLOWER]: Refusing update from remote peer 4b9d6db2bff04d58a4501c86f7b16ae1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:09.957816 3997 consensus_queue.cc:1035] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f29c5901336447f6b31889b46b402bae" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:09.957932 3994 consensus_queue.cc:1035] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4acfd745c5d04219a77068627659653a" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:09.962870 3293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:09.963009 3293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:09.963819 3293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0cb7a257adff4225bfb5098b523b5772: Sending DeleteTablet for 3 replicas of tablet 86ce628e8fc44c459424687fda76ab21
I20260807 06:42:09.964205 3490 tablet_service.cc:1514] Processing DeleteTablet for tablet 86ce628e8fc44c459424687fda76ab21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:09 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:58030
I20260807 06:42:09.964231 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 86ce628e8fc44c459424687fda76ab21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:09 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:52442
I20260807 06:42:09.964262 3678 tablet_service.cc:1514] Processing DeleteTablet for tablet 86ce628e8fc44c459424687fda76ab21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:09 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:51770
I20260807 06:42:09.964581 4016 tablet_replica.cc:330] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: stopping tablet replica
I20260807 06:42:09.964740 4016 raft_consensus.cc:2229] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:42:09.964841 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 3388
I20260807 06:42:09.977999 4016 raft_consensus.cc:2258] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:09.978626 4017 tablet_replica.cc:330] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: stopping tablet replica
I20260807 06:42:09.978760 4017 raft_consensus.cc:2229] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:09.978870 4017 raft_consensus.cc:2258] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:09.979518 4016 ts_tablet_manager.cc:1905] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:09.979951 4017 ts_tablet_manager.cc:1905] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:09.981668 4016 ts_tablet_manager.cc:1918] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:09.981858 4016 log.cc:1196] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/86ce628e8fc44c459424687fda76ab21
I20260807 06:42:09.981865 4017 ts_tablet_manager.cc:1918] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:09.981931 4017 log.cc:1196] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/86ce628e8fc44c459424687fda76ab21
I20260807 06:42:09.982281 4017 ts_tablet_manager.cc:1939] T 86ce628e8fc44c459424687fda76ab21 P 4acfd745c5d04219a77068627659653a: Deleting consensus metadata
I20260807 06:42:09.982292 4016 ts_tablet_manager.cc:1939] T 86ce628e8fc44c459424687fda76ab21 P 4b9d6db2bff04d58a4501c86f7b16ae1: Deleting consensus metadata
I20260807 06:42:09.982791 3268 catalog_manager.cc:4841] TS 4acfd745c5d04219a77068627659653a (127.11.114.195:34043): tablet 86ce628e8fc44c459424687fda76ab21 (table test-table [id=a42aa66c9c64470a8474a54aad5eca64]) successfully deleted
I20260807 06:42:09.982791 3267 catalog_manager.cc:4841] TS 4b9d6db2bff04d58a4501c86f7b16ae1 (127.11.114.194:39481): tablet 86ce628e8fc44c459424687fda76ab21 (table test-table [id=a42aa66c9c64470a8474a54aad5eca64]) successfully deleted
W20260807 06:42:09.991225 3264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:43739 (error 108)
I20260807 06:42:09.991482 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 3591
I20260807 06:42:09.991773 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:09.991630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:43739 (local address 127.0.0.1:58046)
0807 06:42:09.991685 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:09.991719 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:43739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:42:09.994032 3264 catalog_manager.cc:4584] TS f29c5901336447f6b31889b46b402bae (127.11.114.193:43739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86ce628e8fc44c459424687fda76ab21: Network error: Client connection negotiation failed: client connection to 127.11.114.193:43739: connect: Connection refused (error 111)
I20260807 06:42:10.000484 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 3779
I20260807 06:42:10.009989 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 3241
2026-08-07T06:42:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2166 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T06:42:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:10Z Disabled control of system clock
I20260807 06:42:10.044682 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:42921
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:42921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:10.135680 4022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:10.135891 4022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:10.135912 4022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:10.136965 4022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:10.136999 4022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:10.137018 4022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:10.137032 4022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:10.137044 4022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:10.138396 4022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:42921
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:42921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:10.138587 4022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:10.138816 4022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:10.143548 4030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:10.143561 4027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:10.143656 4022 server_base.cc:1034] running on GCE node
W20260807 06:42:10.143604 4028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:10.143980 4022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:10.144266 4022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:10.145416 4022 hybrid_clock.cc:648] HybridClock initialized: now 1786084930145392 us; error 36 us; skew 500 ppm
I20260807 06:42:10.146612 4022 webserver.cc:460] Webserver started at http://127.11.114.254:32901/ using document root <none> and password file <none>
I20260807 06:42:10.146857 4022 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:10.146904 4022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:10.147084 4022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:10.148139 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "fd30099178254c75b79fcceced49be9f"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "ac31a00cbc70512a8ff61977658caf51"
server_key_iv: "c989de45ad5c2fcdfd8e6cf4f247e4ad"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.148504 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "fd30099178254c75b79fcceced49be9f"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "ac31a00cbc70512a8ff61977658caf51"
server_key_iv: "c989de45ad5c2fcdfd8e6cf4f247e4ad"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.149863 4022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:10.150693 4039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.150887 4022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:10.150980 4022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "fd30099178254c75b79fcceced49be9f"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "ac31a00cbc70512a8ff61977658caf51"
server_key_iv: "c989de45ad5c2fcdfd8e6cf4f247e4ad"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.151067 4022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:10.174203 4022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:10.174543 4022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:10.174703 4022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:10.178758 4022 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:42921
I20260807 06:42:10.178791 4112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:42921 every 8 connection(s)
I20260807 06:42:10.179152 4022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:10.180161 4113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:10.182204 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 4022
I20260807 06:42:10.182312 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:10.182521 11723 external_mini_cluster.cc:1428] Setting key 861b8a26965a7b00a5dc335d4fa6857b
I20260807 06:42:10.182780 4113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f: Bootstrap starting.
I20260807 06:42:10.183537 4113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:10.183879 4113 log.cc:824] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:10.184736 4113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f: No bootstrap required, opened a new log
I20260807 06:42:10.186976 4113 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30099178254c75b79fcceced49be9f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42921 } }
I20260807 06:42:10.187150 4113 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:10.187186 4113 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd30099178254c75b79fcceced49be9f, State: Initialized, Role: FOLLOWER
I20260807 06:42:10.187317 4113 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30099178254c75b79fcceced49be9f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42921 } }
I20260807 06:42:10.187405 4113 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:10.187438 4113 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:10.187530 4113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:10.188126 4113 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30099178254c75b79fcceced49be9f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42921 } }
I20260807 06:42:10.188244 4113 leader_election.cc:302] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd30099178254c75b79fcceced49be9f; no voters:
I20260807 06:42:10.188416 4113 leader_election.cc:288] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:10.188490 4122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:10.188704 4122 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [term 1 LEADER]: Becoming Leader. State: Replica: fd30099178254c75b79fcceced49be9f, State: Running, Role: LEADER
I20260807 06:42:10.188719 4113 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:10.189038 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.183086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47662 (local address 127.11.114.254:42921)
0807 06:42:10.183313 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.183322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.183354 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:42:10.183961 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.183970 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.184054 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.184199 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:10.184456 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.185885 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.186652 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.187276 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.187399 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.188138 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:10.188142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:10.188142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:10.188362 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:10.188406 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:10.188421 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:10.188494 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.188739 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.188836 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260807 06:42:10.190322 4122 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30099178254c75b79fcceced49be9f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42921 } }
I20260807 06:42:10.191018 4124 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd30099178254c75b79fcceced49be9f. Latest consensus state: current_term: 1 leader_uuid: "fd30099178254c75b79fcceced49be9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30099178254c75b79fcceced49be9f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42921 } } }
I20260807 06:42:10.191088 4124 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:10.190912 4123 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd30099178254c75b79fcceced49be9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30099178254c75b79fcceced49be9f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42921 } } }
I20260807 06:42:10.191252 4123 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:10.191362 4142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:10.191762 4142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:10.193193 4142 catalog_manager.cc:1347] Generated new cluster ID: c8e5ae622a774fdaae4afe6c36c9c420
I20260807 06:42:10.193248 4142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:10.200594 4142 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:10.201146 4142 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:10.204685 4142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f: Generated new TSK 0
I20260807 06:42:10.204861 4142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:10.209879 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:42921
--builtin_ntp_servers=127.11.114.212:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:10.293229 4146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:10.293438 4146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:10.293462 4146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:10.294649 4146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:10.294692 4146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:10.294718 4146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:42:10.296058 4146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:42921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:10.296314 4146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:10.296630 4146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:10.302583 4156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:10.302682 4158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:10.302592 4155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:10.302917 4146 server_base.cc:1034] running on GCE node
I20260807 06:42:10.303148 4146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:10.303412 4146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:10.304627 4146 hybrid_clock.cc:648] HybridClock initialized: now 1786084930304609 us; error 31 us; skew 500 ppm
I20260807 06:42:10.306027 4146 webserver.cc:460] Webserver started at http://127.11.114.193:32775/ using document root <none> and password file <none>
I20260807 06:42:10.306327 4146 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:10.306401 4146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:10.306552 4146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:10.307631 4146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "6cd522a5b4da46ce8dde9ecda7c81a46"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "2d860d53b26a0ea51c8b9755110ecdc3"
server_key_iv: "aa866def791f30f5dc473463a48fa67d"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.307991 4146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "6cd522a5b4da46ce8dde9ecda7c81a46"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "2d860d53b26a0ea51c8b9755110ecdc3"
server_key_iv: "aa866def791f30f5dc473463a48fa67d"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.309396 4146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:42:10.310293 4168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.310494 4146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:10.310622 4146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "6cd522a5b4da46ce8dde9ecda7c81a46"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "2d860d53b26a0ea51c8b9755110ecdc3"
server_key_iv: "aa866def791f30f5dc473463a48fa67d"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.310709 4146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:10.327855 4146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:10.328231 4146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:10.328404 4146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:10.328644 4146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:10.329010 4146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:10.329069 4146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.329121 4146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:10.329154 4146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.335172 4146 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:41261
I20260807 06:42:10.335212 4332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:41261 every 8 connection(s)
I20260807 06:42:10.335600 4146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:42:10.341149 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.336422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:33295 (local address 127.11.114.254:42921)
0807 06:42:10.336472 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.336473 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.336731 (+ 258us) server_negotiation.cc:407] Connection header received
0807 06:42:10.337286 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.337287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.337300 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.337360 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:10.338160 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.338683 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.339725 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.339994 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.340070 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.340908 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:10.340909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:10.340910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:10.340914 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:10.340928 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:10.340933 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:10.340976 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.341063 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.341087 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:42:10.341418 4336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.336385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:42921 (local address 127.11.114.193:33295)
0807 06:42:10.336599 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.336632 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:10.337168 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:10.337438 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:10.337442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:10.337461 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:10.338057 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.338062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.338805 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.338807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.339627 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.339629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.340122 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.340123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.340247 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.340768 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:10.340769 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:10.340797 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:10.340820 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:10.340989 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:10.340994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:10.340996 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:42:10.341050 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 06:42:10.341142 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260807 06:42:10.341990 4333 heartbeater.cc:347] Connected to a master server at 127.11.114.254:42921
I20260807 06:42:10.342103 4333 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:10.342310 4333 heartbeater.cc:511] Master 127.11.114.254:42921 requested a full tablet report, sending...
I20260807 06:42:10.342888 4064 ts_manager.cc:194] Registered new tserver with Master: 6cd522a5b4da46ce8dde9ecda7c81a46 (127.11.114.193:41261)
I20260807 06:42:10.343523 4064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.193:33295
I20260807 06:42:10.345752 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 4146
I20260807 06:42:10.345865 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:42:10.346081 11723 external_mini_cluster.cc:1428] Setting key 07ac27799840248f36a1bd7f3b24e7e9
I20260807 06:42:10.347016 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:42921
--builtin_ntp_servers=127.11.114.212:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:10.433616 4337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:10.433830 4337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:10.433851 4337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:10.434988 4337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:10.435030 4337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:10.435056 4337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:42:10.436389 4337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:42921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:10.436635 4337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:10.436933 4337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:10.442212 4347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:10.442335 4337 server_base.cc:1034] running on GCE node
W20260807 06:42:10.442255 4344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:10.442441 4350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:10.442663 4337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:10.442870 4337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:10.444036 4337 hybrid_clock.cc:648] HybridClock initialized: now 1786084930444018 us; error 43 us; skew 500 ppm
I20260807 06:42:10.445370 4337 webserver.cc:460] Webserver started at http://127.11.114.194:37499/ using document root <none> and password file <none>
I20260807 06:42:10.445683 4337 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:10.445736 4337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:10.445891 4337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:10.447013 4337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "dc262ddfc82a4b749dc2aa67e5027996"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "61be35dc4c230e4df9b5f467048b6086"
server_key_iv: "1013dc995283c0712e358feb6f4280bf"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.447377 4337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "dc262ddfc82a4b749dc2aa67e5027996"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "61be35dc4c230e4df9b5f467048b6086"
server_key_iv: "1013dc995283c0712e358feb6f4280bf"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.448680 4337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:10.449414 4358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.449625 4337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:10.449726 4337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "dc262ddfc82a4b749dc2aa67e5027996"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "61be35dc4c230e4df9b5f467048b6086"
server_key_iv: "1013dc995283c0712e358feb6f4280bf"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.449810 4337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:10.460529 4337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:10.460812 4337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:10.460963 4337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:10.461189 4337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:10.461509 4337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:10.461597 4337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.461632 4337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:10.461659 4337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.467480 4337 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:43557
I20260807 06:42:10.467519 4522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:43557 every 8 connection(s)
I20260807 06:42:10.467857 4337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:42:10.472572 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 4337
I20260807 06:42:10.472694 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:42:10.472927 11723 external_mini_cluster.cc:1428] Setting key 4b941ff666092467d39fde4d2ea14aac
I20260807 06:42:10.473331 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.468616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:37533 (local address 127.11.114.254:42921)
0807 06:42:10.468681 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.468683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.468885 (+ 202us) server_negotiation.cc:407] Connection header received
0807 06:42:10.469435 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.469437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.469451 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.469514 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:10.470207 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.470708 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.471701 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.471905 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.472014 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.472970 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:10.472971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:10.472971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:10.472977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:10.472998 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:10.473019 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:10.473070 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.473202 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.473229 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:42:10.473655 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.468570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:42921 (local address 127.11.114.194:37533)
0807 06:42:10.468761 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.468792 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:10.469324 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:10.469613 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:10.469617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:10.469625 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:10.470106 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.470111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.470824 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.470825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.471604 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.471607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.472023 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.472025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.472157 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.472831 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:10.472832 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:10.472860 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:10.472882 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:10.473081 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:10.473090 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:10.473100 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 06:42:10.473185 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 06:42:10.473332 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260807 06:42:10.474030 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:42921
--builtin_ntp_servers=127.11.114.212:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:42:10.475584 4523 heartbeater.cc:347] Connected to a master server at 127.11.114.254:42921
I20260807 06:42:10.475669 4523 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:10.475863 4523 heartbeater.cc:511] Master 127.11.114.254:42921 requested a full tablet report, sending...
I20260807 06:42:10.476282 4064 ts_manager.cc:194] Registered new tserver with Master: dc262ddfc82a4b749dc2aa67e5027996 (127.11.114.194:43557)
I20260807 06:42:10.476665 4064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.194:37533
W20260807 06:42:10.556929 4528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:10.557137 4528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:10.557156 4528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:10.558254 4528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:10.558287 4528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:10.558311 4528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:42:10.559588 4528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:42921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:10.559784 4528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:10.560034 4528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:10.565027 4537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:10.565116 4543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:10.565029 4528 server_base.cc:1034] running on GCE node
W20260807 06:42:10.565016 4536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:10.565474 4528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:10.565744 4528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:10.566917 4528 hybrid_clock.cc:648] HybridClock initialized: now 1786084930566906 us; error 28 us; skew 500 ppm
I20260807 06:42:10.568161 4528 webserver.cc:460] Webserver started at http://127.11.114.195:34189/ using document root <none> and password file <none>
I20260807 06:42:10.568418 4528 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:10.568463 4528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:10.568638 4528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:10.569798 4528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "df461f492f4142b89fb2f538adbb0d6f"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "8ca1907fcdb013430538e8ac9f808491"
server_key_iv: "60aefcf7102761aef783fbda50acb561"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.570159 4528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "df461f492f4142b89fb2f538adbb0d6f"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "8ca1907fcdb013430538e8ac9f808491"
server_key_iv: "60aefcf7102761aef783fbda50acb561"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.571448 4528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:42:10.572266 4555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.572474 4528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:10.572573 4528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "df461f492f4142b89fb2f538adbb0d6f"
format_stamp: "Formatted at 2026-08-07 06:42:10 on dist-test-slave-gkw7"
server_key: "8ca1907fcdb013430538e8ac9f808491"
server_key_iv: "60aefcf7102761aef783fbda50acb561"
server_key_version: "encryptionkey@0"
I20260807 06:42:10.572652 4528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:10.596771 4528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:10.597097 4528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:10.597252 4528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:10.597482 4528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:10.597856 4528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:10.597891 4528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.597914 4528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:10.597968 4528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:10.603690 4528 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:36533
I20260807 06:42:10.603732 4718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:36533 every 8 connection(s)
I20260807 06:42:10.604074 4528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:42:10.609123 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.604804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:32903 (local address 127.11.114.254:42921)
0807 06:42:10.604879 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.604882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.605147 (+ 265us) server_negotiation.cc:407] Connection header received
0807 06:42:10.605692 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.605693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.605704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.605770 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:10.606437 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.606930 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.607864 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.608046 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.608096 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.608881 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:10.608883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:10.608883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:10.608888 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:10.608901 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:10.608905 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:10.608947 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.609038 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.609062 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:42:10.609316 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.604794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:42921 (local address 127.11.114.195:32903)
0807 06:42:10.605016 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.605051 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:10.605590 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:10.605828 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:10.605831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:10.605840 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:10.606364 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.606368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.607010 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.607011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.607787 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.607789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.608121 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.608121 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.608231 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.608748 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:10.608748 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:10.608776 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:10.608797 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:10.608964 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:10.608968 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:10.608971 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:42:10.609023 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:42:10.609103 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260807 06:42:10.609863 4719 heartbeater.cc:347] Connected to a master server at 127.11.114.254:42921
I20260807 06:42:10.609977 4719 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:10.610085 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 4528
I20260807 06:42:10.610198 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:42:10.610212 4719 heartbeater.cc:511] Master 127.11.114.254:42921 requested a full tablet report, sending...
I20260807 06:42:10.610463 11723 external_mini_cluster.cc:1428] Setting key a68bba55e79a39692f12c286b5aaaebb
I20260807 06:42:10.610656 4064 ts_manager.cc:194] Registered new tserver with Master: df461f492f4142b89fb2f538adbb0d6f (127.11.114.195:36533)
I20260807 06:42:10.611088 4064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.195:32903
I20260807 06:42:10.611311 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:42:10.615299 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.612668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.11.114.254:42921)
0807 06:42:10.612723 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.612725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.612817 (+ 92us) server_negotiation.cc:407] Connection header received
0807 06:42:10.612885 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.612886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.612894 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.612928 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:10.613062 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.613501 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.614253 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.614428 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.614481 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.615121 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:10.615122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:10.615123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:10.615126 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:10.615138 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:10.615142 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:10.615184 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.615229 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.615248 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:42:10.617102 4063 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:42:10.618788 4063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:42:10.625449 4737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.621060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57854 (local address 127.11.114.195:36533)
0807 06:42:10.621280 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.621285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.621298 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:42:10.621403 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.621406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.621420 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.621516 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:10.621710 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.622866 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.624372 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.625057 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.625130 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.625198 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.625330 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.625368 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":115,"threads_started":1}
I20260807 06:42:10.625466 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.620949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:43557 (local address 127.0.0.1:60400)
0807 06:42:10.621173 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.621216 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:10.621291 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:10.621730 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:10.621730 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:10.621732 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:10.621768 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.621769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.623002 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.623004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.624086 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.624088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.625121 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.625122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.625207 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.625225 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 06:42:10.625304 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 06:42:10.625367 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":93,"thread_start_us":60,"threads_started":1}
I20260807 06:42:10.625783 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.621044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.11.114.194:43557)
0807 06:42:10.621376 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.621380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.621389 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:10.621440 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.621442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.621468 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.621608 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:10.621867 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.622941 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.624204 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.624977 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.625061 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.625157 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.625648 (+ 491us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.625703 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":108,"threads_started":1}
I20260807 06:42:10.625466 4733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.620984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:36533 (local address 127.0.0.1:57854)
0807 06:42:10.621250 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.621259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:10.621301 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:10.621520 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:10.621524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:10.621525 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:10.621633 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.621634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.622964 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.622965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.624287 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.624290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.625135 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.625136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.625207 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.625225 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 06:42:10.625304 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 06:42:10.625367 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":125,"thread_start_us":109,"threads_started":1}
I20260807 06:42:10.626523 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.621182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:41261 (local address 127.0.0.1:36026)
0807 06:42:10.621624 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.621634 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:10.621680 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:10.621997 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:10.621998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:10.622000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:10.622094 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.622096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.623000 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.623001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.624288 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.624294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.626261 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.626262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.626326 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.626331 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:10.626400 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 06:42:10.626444 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":47,"thread_start_us":78,"threads_started":1}
I20260807 06:42:10.626622 4423 tablet_service.cc:1467] Processing CreateTablet for tablet 6a179a173b5741f7b89c0b67ce9d57c7 (DEFAULT_TABLE table=test-table [id=db683ee4e8774a8e9c67abeb6fd514b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:10.626768 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.621392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36026 (local address 127.11.114.193:41261)
0807 06:42:10.621830 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.621835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.621845 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:42:10.621882 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.621884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.621899 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.622182 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:10.622226 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.622884 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.625233 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.626147 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.626260 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.626346 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.626648 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.626693 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":85,"threads_started":1}
I20260807 06:42:10.627043 4423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a179a173b5741f7b89c0b67ce9d57c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:10.626974 4622 tablet_service.cc:1467] Processing CreateTablet for tablet 6a179a173b5741f7b89c0b67ce9d57c7 (DEFAULT_TABLE table=test-table [id=db683ee4e8774a8e9c67abeb6fd514b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:10.627215 4622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a179a173b5741f7b89c0b67ce9d57c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:10.627396 4238 tablet_service.cc:1467] Processing CreateTablet for tablet 6a179a173b5741f7b89c0b67ce9d57c7 (DEFAULT_TABLE table=test-table [id=db683ee4e8774a8e9c67abeb6fd514b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:10.627705 4238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a179a173b5741f7b89c0b67ce9d57c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:10.629649 4740 tablet_bootstrap.cc:492] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: Bootstrap starting.
I20260807 06:42:10.630264 4740 tablet_bootstrap.cc:654] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:10.630640 4740 log.cc:824] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:10.631031 4745 tablet_bootstrap.cc:492] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46: Bootstrap starting.
I20260807 06:42:10.631179 4741 tablet_bootstrap.cc:492] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Bootstrap starting.
I20260807 06:42:10.631532 4740 tablet_bootstrap.cc:492] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: No bootstrap required, opened a new log
I20260807 06:42:10.631616 4740 ts_tablet_manager.cc:1397] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:42:10.631762 4745 tablet_bootstrap.cc:654] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:10.631843 4741 tablet_bootstrap.cc:654] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:10.632078 4745 log.cc:824] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:10.632164 4741 log.cc:824] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:10.632798 4745 tablet_bootstrap.cc:492] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46: No bootstrap required, opened a new log
I20260807 06:42:10.632889 4745 ts_tablet_manager.cc:1397] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:42:10.632965 4741 tablet_bootstrap.cc:492] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: No bootstrap required, opened a new log
I20260807 06:42:10.633044 4741 ts_tablet_manager.cc:1397] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:42:10.634444 4740 raft_consensus.cc:348] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.634629 4740 raft_consensus.cc:374] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:10.634668 4740 raft_consensus.cc:729] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df461f492f4142b89fb2f538adbb0d6f, State: Initialized, Role: FOLLOWER
I20260807 06:42:10.634815 4740 consensus_queue.cc:260] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.635085 4740 ts_tablet_manager.cc:1428] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260807 06:42:10.635030 4745 raft_consensus.cc:348] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.635221 4719 heartbeater.cc:503] Master 127.11.114.254:42921 was elected leader, sending a full tablet report...
I20260807 06:42:10.635222 4745 raft_consensus.cc:374] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:10.635262 4745 raft_consensus.cc:729] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd522a5b4da46ce8dde9ecda7c81a46, State: Initialized, Role: FOLLOWER
I20260807 06:42:10.635381 4745 consensus_queue.cc:260] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.635622 4745 ts_tablet_manager.cc:1428] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:10.635679 4741 raft_consensus.cc:348] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.635799 4333 heartbeater.cc:503] Master 127.11.114.254:42921 was elected leader, sending a full tablet report...
I20260807 06:42:10.635860 4741 raft_consensus.cc:374] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:10.635907 4741 raft_consensus.cc:729] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc262ddfc82a4b749dc2aa67e5027996, State: Initialized, Role: FOLLOWER
I20260807 06:42:10.636059 4741 consensus_queue.cc:260] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.636322 4741 ts_tablet_manager.cc:1428] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 06:42:10.636479 4523 heartbeater.cc:503] Master 127.11.114.254:42921 was elected leader, sending a full tablet report...
I20260807 06:42:10.716588 4754 raft_consensus.cc:482] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:42:10.716722 4754 raft_consensus.cc:504] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.717170 4754 leader_election.cc:288] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df461f492f4142b89fb2f538adbb0d6f (127.11.114.195:36533), 6cd522a5b4da46ce8dde9ecda7c81a46 (127.11.114.193:41261)
W20260807 06:42:10.718362 4524 tablet.cc:2369] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:42:10.721086 4737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.717510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:35695 (local address 127.11.114.195:36533)
0807 06:42:10.717579 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.717582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.717700 (+ 118us) server_negotiation.cc:407] Connection header received
0807 06:42:10.717843 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.717844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.717858 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.717914 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:10.718106 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.718782 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.720141 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.720697 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.720774 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.720783 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.720972 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.721011 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:42:10.721086 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.717468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:50559 (local address 127.11.114.193:41261)
0807 06:42:10.717588 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.717590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.717600 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:42:10.717632 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.717633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.717647 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.717726 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:10.717944 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.718718 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.720142 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.720696 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.720774 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.720783 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.720972 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.721011 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 06:42:10.721098 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.717381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:41261 (local address 127.11.114.194:50559)
0807 06:42:10.717426 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.717438 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:10.717484 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:10.717755 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:10.717756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:10.717759 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:10.717837 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.717839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.718802 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.718802 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.720043 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.720046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.720794 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.720795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.720886 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.720891 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:10.720950 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:42:10.721011 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":10}
I20260807 06:42:10.721104 4755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.717383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:36533 (local address 127.11.114.194:35695)
0807 06:42:10.717642 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.717652 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:10.717727 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:10.717952 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:10.717953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:10.717956 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:10.718029 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.718030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.718854 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.718855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.720043 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:10.720046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.720795 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.720795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:10.720886 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.720891 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:10.720950 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:42:10.721009 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":85,"thread_start_us":140,"threads_started":1}
I20260807 06:42:10.721522 4266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a179a173b5741f7b89c0b67ce9d57c7" candidate_uuid: "dc262ddfc82a4b749dc2aa67e5027996" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" is_pre_election: true
I20260807 06:42:10.721738 4266 raft_consensus.cc:2454] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc262ddfc82a4b749dc2aa67e5027996 in term 0.
I20260807 06:42:10.721915 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a179a173b5741f7b89c0b67ce9d57c7" candidate_uuid: "dc262ddfc82a4b749dc2aa67e5027996" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df461f492f4142b89fb2f538adbb0d6f" is_pre_election: true
I20260807 06:42:10.722023 4359 leader_election.cc:302] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd522a5b4da46ce8dde9ecda7c81a46, dc262ddfc82a4b749dc2aa67e5027996; no voters:
I20260807 06:42:10.722067 4653 raft_consensus.cc:2454] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc262ddfc82a4b749dc2aa67e5027996 in term 0.
I20260807 06:42:10.722149 4754 raft_consensus.cc:2781] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:42:10.722224 4754 raft_consensus.cc:482] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:42:10.722265 4754 raft_consensus.cc:3037] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:10.722998 4754 raft_consensus.cc:504] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.723153 4754 leader_election.cc:288] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [CANDIDATE]: Term 1 election: Requested vote from peers df461f492f4142b89fb2f538adbb0d6f (127.11.114.195:36533), 6cd522a5b4da46ce8dde9ecda7c81a46 (127.11.114.193:41261)
I20260807 06:42:10.723294 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a179a173b5741f7b89c0b67ce9d57c7" candidate_uuid: "dc262ddfc82a4b749dc2aa67e5027996" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df461f492f4142b89fb2f538adbb0d6f"
I20260807 06:42:10.723372 4653 raft_consensus.cc:3037] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:10.723347 4266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a179a173b5741f7b89c0b67ce9d57c7" candidate_uuid: "dc262ddfc82a4b749dc2aa67e5027996" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46"
I20260807 06:42:10.723429 4266 raft_consensus.cc:3037] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:10.724032 4653 raft_consensus.cc:2454] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc262ddfc82a4b749dc2aa67e5027996 in term 1.
I20260807 06:42:10.724223 4363 leader_election.cc:302] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc262ddfc82a4b749dc2aa67e5027996, df461f492f4142b89fb2f538adbb0d6f; no voters:
I20260807 06:42:10.724267 4266 raft_consensus.cc:2454] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc262ddfc82a4b749dc2aa67e5027996 in term 1.
I20260807 06:42:10.724354 4754 raft_consensus.cc:2781] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:10.724517 4754 raft_consensus.cc:686] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 1 LEADER]: Becoming Leader. State: Replica: dc262ddfc82a4b749dc2aa67e5027996, State: Running, Role: LEADER
I20260807 06:42:10.724635 4754 consensus_queue.cc:237] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } }
I20260807 06:42:10.725502 4064 catalog_manager.cc:5495] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc262ddfc82a4b749dc2aa67e5027996 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "dc262ddfc82a4b749dc2aa67e5027996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc262ddfc82a4b749dc2aa67e5027996" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 43557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260807 06:42:10.745517 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.743331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60406 (local address 127.11.114.194:43557)
0807 06:42:10.743385 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:10.743387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:10.743424 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:42:10.743460 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:10.743461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:10.743479 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:10.743535 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:10.743657 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.744140 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.744858 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:10.745044 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:10.745099 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:10.745231 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:10.745328 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:10.745388 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:10.745424 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 06:42:10.745439 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:42:10.747437 4653 raft_consensus.cc:1261] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 1 FOLLOWER]: Refusing update from remote peer dc262ddfc82a4b749dc2aa67e5027996: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:10.747478 4266 raft_consensus.cc:1261] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 1 FOLLOWER]: Refusing update from remote peer dc262ddfc82a4b749dc2aa67e5027996: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:10.747807 4756 consensus_queue.cc:1035] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df461f492f4142b89fb2f538adbb0d6f" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 36533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:10.747963 4754 consensus_queue.cc:1035] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd522a5b4da46ce8dde9ecda7c81a46" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:10.751065 4762 mvcc.cc:205] Tried to move back new op lower bound from 7315803876338315264 to 7315803876248739840. Current Snapshot: MvccSnapshot[applied={T|T < 7315803876338315264 or (T in {7315803876338315264})}]
I20260807 06:42:10.753226 4064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:10.753330 4064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:10.754119 4064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd30099178254c75b79fcceced49be9f: Sending DeleteTablet for 3 replicas of tablet 6a179a173b5741f7b89c0b67ce9d57c7
I20260807 06:42:10.754416 4423 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a179a173b5741f7b89c0b67ce9d57c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:10 UTC) from {username='slave'} at 127.0.0.1:60400
I20260807 06:42:10.754490 4622 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a179a173b5741f7b89c0b67ce9d57c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:10 UTC) from {username='slave'} at 127.0.0.1:57854
I20260807 06:42:10.754714 4772 tablet_replica.cc:330] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: stopping tablet replica
I20260807 06:42:10.754752 4238 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a179a173b5741f7b89c0b67ce9d57c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:10 UTC) from {username='slave'} at 127.0.0.1:36026
I20260807 06:42:10.754827 4772 raft_consensus.cc:2229] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:42:10.754958 4772 raft_consensus.cc:2258] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:10.754966 4775 tablet_replica.cc:330] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46: stopping tablet replica
I20260807 06:42:10.755059 4775 raft_consensus.cc:2229] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:10.755218 4775 raft_consensus.cc:2258] T 6a179a173b5741f7b89c0b67ce9d57c7 P 6cd522a5b4da46ce8dde9ecda7c81a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:10.755280 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 4146
I20260807 06:42:10.755493 4772 ts_tablet_manager.cc:1905] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:10.757800 4772 ts_tablet_manager.cc:1918] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:10.757869 4772 log.cc:1196] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/6a179a173b5741f7b89c0b67ce9d57c7
I20260807 06:42:10.758122 4772 ts_tablet_manager.cc:1939] T 6a179a173b5741f7b89c0b67ce9d57c7 P dc262ddfc82a4b749dc2aa67e5027996: Deleting consensus metadata
I20260807 06:42:10.758518 4040 catalog_manager.cc:4841] TS dc262ddfc82a4b749dc2aa67e5027996 (127.11.114.194:43557): tablet 6a179a173b5741f7b89c0b67ce9d57c7 (table test-table [id=db683ee4e8774a8e9c67abeb6fd514b4]) successfully deleted
I20260807 06:42:10.765690 4774 tablet_replica.cc:330] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: stopping tablet replica
I20260807 06:42:10.765836 4774 raft_consensus.cc:2229] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:10.765959 4774 raft_consensus.cc:2258] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:10.766446 4774 ts_tablet_manager.cc:1905] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:10.768067 4774 ts_tablet_manager.cc:1918] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:10.768185 4774 log.cc:1196] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/6a179a173b5741f7b89c0b67ce9d57c7
I20260807 06:42:10.768461 4774 ts_tablet_manager.cc:1939] T 6a179a173b5741f7b89c0b67ce9d57c7 P df461f492f4142b89fb2f538adbb0d6f: Deleting consensus metadata
I20260807 06:42:10.768893 4046 catalog_manager.cc:4841] TS df461f492f4142b89fb2f538adbb0d6f (127.11.114.195:36533): tablet 6a179a173b5741f7b89c0b67ce9d57c7 (table test-table [id=db683ee4e8774a8e9c67abeb6fd514b4]) successfully deleted
W20260807 06:42:10.769147 4040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:41261 (error 108)
I20260807 06:42:10.769304 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 4337
I20260807 06:42:10.781179 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:10.780997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:41261 (local address 127.0.0.1:36028)
0807 06:42:10.781063 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:10.781127 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:41261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 06:42:10.781436 4040 catalog_manager.cc:4584] TS 6cd522a5b4da46ce8dde9ecda7c81a46 (127.11.114.193:41261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a179a173b5741f7b89c0b67ce9d57c7: Network error: Client connection negotiation failed: client connection to 127.11.114.193:41261: connect: Connection refused (error 111)
I20260807 06:42:10.792990 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 4528
I20260807 06:42:10.801698 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 4022
2026-08-07T06:42:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (789 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:10 dist-test-slave-gkw7 krb5kdc[4780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:10 dist-test-slave-gkw7 krb5kdc[4780](info): set up 2 sockets
Aug 07 06:42:10 dist-test-slave-gkw7 krb5kdc[4780](info): commencing operation
krb5kdc: starting...
W20260807 06:42:12.011682 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:12.147796 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:41031
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:41031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:12.230486 4797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:12.230700 4797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:12.230723 4797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:12.231886 4797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:12.231930 4797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:12.231945 4797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:12.231957 4797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:12.232010 4797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:12.233475 4797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:41031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:41031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:12.233752 4797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:12.234009 4797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:12.239161 4803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:12.239179 4806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:12.239125 4804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:12.239213 4797 server_base.cc:1034] running on GCE node
I20260807 06:42:12.239711 4797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:12.240018 4797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:12.241194 4797 hybrid_clock.cc:648] HybridClock initialized: now 1786084932241170 us; error 38 us; skew 500 ppm
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:12.243968 4797 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:12.244560 4797 webserver.cc:460] Webserver started at http://127.11.114.254:45657/ using document root <none> and password file <none>
I20260807 06:42:12.244812 4797 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:12.244886 4797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:12.245064 4797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:12.246305 4797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "82002df171ca4ea59d75d00058457d54"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "bf11f8862032db375700c47d9f923d2f"
server_key_iv: "28ad7f7a21e79ce28b55be9a7499345d"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.246716 4797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "82002df171ca4ea59d75d00058457d54"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "bf11f8862032db375700c47d9f923d2f"
server_key_iv: "28ad7f7a21e79ce28b55be9a7499345d"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.248176 4797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:42:12.248973 4815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.249155 4797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.249270 4797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "82002df171ca4ea59d75d00058457d54"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "bf11f8862032db375700c47d9f923d2f"
server_key_iv: "28ad7f7a21e79ce28b55be9a7499345d"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.249352 4797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:12.263199 4797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:12.265792 4797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:12.265980 4797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:12.270470 4797 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:41031
I20260807 06:42:12.270507 4893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:41031 every 8 connection(s)
I20260807 06:42:12.270907 4797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:12.271812 4894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:12.273300 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 4797
I20260807 06:42:12.273423 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:12.273674 11723 external_mini_cluster.cc:1428] Setting key 953bd2ac0a18f11d7d2aee57b5b81705
I20260807 06:42:12.274914 4894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54: Bootstrap starting.
I20260807 06:42:12.275691 4894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:12.276050 4894 log.cc:824] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:12.276834 4894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54: No bootstrap required, opened a new log
I20260807 06:42:12.279263 4894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82002df171ca4ea59d75d00058457d54" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41031 } }
I20260807 06:42:12.279449 4894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:12.279495 4894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82002df171ca4ea59d75d00058457d54, State: Initialized, Role: FOLLOWER
I20260807 06:42:12.279635 4894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82002df171ca4ea59d75d00058457d54" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41031 } }
I20260807 06:42:12.279728 4894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:12.279762 4894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:12.279819 4894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:12.280599 4894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82002df171ca4ea59d75d00058457d54" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41031 } }
I20260807 06:42:12.280694 4894 leader_election.cc:302] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82002df171ca4ea59d75d00058457d54; no voters:
I20260807 06:42:12.280884 4894 leader_election.cc:288] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:12.280906 4901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:12.281023 4901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [term 1 LEADER]: Becoming Leader. State: Replica: 82002df171ca4ea59d75d00058457d54, State: Running, Role: LEADER
I20260807 06:42:12.281155 4894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:12.281172 4901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82002df171ca4ea59d75d00058457d54" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41031 } }
I20260807 06:42:12.281558 4901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82002df171ca4ea59d75d00058457d54. Latest consensus state: current_term: 1 leader_uuid: "82002df171ca4ea59d75d00058457d54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82002df171ca4ea59d75d00058457d54" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41031 } } }
I20260807 06:42:12.281708 4901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:12.282210 4910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:12.282455 4902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82002df171ca4ea59d75d00058457d54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82002df171ca4ea59d75d00058457d54" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41031 } } }
I20260807 06:42:12.282883 4910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:12.283532 4902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:12.284618 4910 catalog_manager.cc:1347] Generated new cluster ID: 64eed4b9a9724e80b7e8523119946a47
I20260807 06:42:12.284674 4910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:12.285995 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.274236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.11.114.254:41031)
0807 06:42:12.274424 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.274432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.274457 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:42:12.274923 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.274929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.275007 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.275141 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:12.276009 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.277176 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.278325 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.278739 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.278828 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.281912 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:12.281917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:12.281917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:12.281930 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:12.284223 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.284664 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.284667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.284668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.284812 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.285051 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.285052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.285052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.285360 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:12.285559 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.285648 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.285795 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260807 06:42:12.293984 4910 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:12.294466 4910 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:12.299600 4910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54: Generated new TSK 0
I20260807 06:42:12.299799 4910 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:42:12.326397 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:41031
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:12.409901 4930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:12.410105 4930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:12.410127 4930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:12.411283 4930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:12.411330 4930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:12.411356 4930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:42:12.412737 4930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:41031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:12.412923 4930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:12.413100 4930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:12.418146 4937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:12.418134 4938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:12.418292 4930 server_base.cc:1034] running on GCE node
W20260807 06:42:12.418136 4940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:12.418586 4930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:12.418839 4930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:12.420027 4930 hybrid_clock.cc:648] HybridClock initialized: now 1786084932420013 us; error 51 us; skew 500 ppm
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:12.422706 4930 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:42:12.423137 4930 webserver.cc:460] Webserver started at http://127.11.114.193:34873/ using document root <none> and password file <none>
I20260807 06:42:12.423375 4930 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:12.423444 4930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:12.423600 4930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:12.424762 4930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "36281f21209a45f9a7a8a76903aa1bf7"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "0c4f8867113a80dca3ebdca6c0680fe6"
server_key_iv: "73b6c848dd88bb0139be19e4693f7325"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.425128 4930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "36281f21209a45f9a7a8a76903aa1bf7"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "0c4f8867113a80dca3ebdca6c0680fe6"
server_key_iv: "73b6c848dd88bb0139be19e4693f7325"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.426594 4930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:12.427335 4949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.427507 4930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:42:12.427618 4930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "36281f21209a45f9a7a8a76903aa1bf7"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "0c4f8867113a80dca3ebdca6c0680fe6"
server_key_iv: "73b6c848dd88bb0139be19e4693f7325"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.427692 4930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:12.442744 4930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:12.445089 4930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:12.445261 4930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:12.445501 4930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:12.445847 4930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:12.445906 4930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.445947 4930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:12.445976 4930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.451700 4930 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:34067
I20260807 06:42:12.451790 5100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:34067 every 8 connection(s)
I20260807 06:42:12.452114 4930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:12.461059 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.452847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:35743 (local address 127.11.114.254:41031)
0807 06:42:12.452914 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.452916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.453152 (+ 236us) server_negotiation.cc:407] Connection header received
0807 06:42:12.453794 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.453795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.453806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.453863 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:12.454789 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.455303 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.456246 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.456461 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.456513 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.458608 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:12.458609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:12.458610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:12.458617 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:12.459895 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.460267 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.460267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.460268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.460321 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.460532 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.460532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.460533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.460636 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:12.460686 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.460922 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.460993 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:42:12.461215 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.452844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:41031 (local address 127.11.114.193:35743)
0807 06:42:12.452996 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.453031 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:12.453687 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:12.453939 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:12.453942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:12.454511 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:12.454702 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.454707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.455383 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.455384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.456170 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.456172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.456535 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.456536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.456636 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.457099 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:12.457107 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:12.458520 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:12.459971 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:12.459978 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:12.459980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:12.460194 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:12.460379 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:12.460380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:12.460380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:12.460477 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:12.460692 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:12.460696 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:12.460814 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 06:42:12.460895 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:42:12.461014 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260807 06:42:12.461823 5102 heartbeater.cc:347] Connected to a master server at 127.11.114.254:41031
I20260807 06:42:12.461854 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 4930
I20260807 06:42:12.461936 5102 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:12.461956 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:42:12.462141 5102 heartbeater.cc:511] Master 127.11.114.254:41031 requested a full tablet report, sending...
I20260807 06:42:12.462183 11723 external_mini_cluster.cc:1428] Setting key 2665a24d3b10aaf689c1f68cea4225cc
I20260807 06:42:12.462684 4843 ts_manager.cc:194] Registered new tserver with Master: 36281f21209a45f9a7a8a76903aa1bf7 (127.11.114.193:34067)
I20260807 06:42:12.463315 4843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:35743
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:42:12.489128 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:41031
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:12.572854 5109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:12.573134 5109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:12.573175 5109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:12.574348 5109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:12.574427 5109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:12.574471 5109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:42:12.575932 5109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:41031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:12.576205 5109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:12.576467 5109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:12.581727 5115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:12.581751 5114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:12.581737 5119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:12.582036 5109 server_base.cc:1034] running on GCE node
I20260807 06:42:12.582271 5109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:12.582576 5109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:12.583734 5109 hybrid_clock.cc:648] HybridClock initialized: now 1786084932583713 us; error 40 us; skew 500 ppm
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:12.586266 5109 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:42:12.586777 5109 webserver.cc:460] Webserver started at http://127.11.114.194:41677/ using document root <none> and password file <none>
I20260807 06:42:12.587020 5109 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:12.587090 5109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:12.587229 5109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:12.588311 5109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "d1cfc96ebc244cbb980850369180de8a"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "efe4dfee528c7e66ebb2238a205772ab"
server_key_iv: "020f35655155d4f415e2af832e9f4671"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.588693 5109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "d1cfc96ebc244cbb980850369180de8a"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "efe4dfee528c7e66ebb2238a205772ab"
server_key_iv: "020f35655155d4f415e2af832e9f4671"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.590155 5109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:12.590930 5130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.591226 5109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:12.591348 5109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "d1cfc96ebc244cbb980850369180de8a"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "efe4dfee528c7e66ebb2238a205772ab"
server_key_iv: "020f35655155d4f415e2af832e9f4671"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.591432 5109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:12.612160 5109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:12.614543 5109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:12.614727 5109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:12.615001 5109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:12.615398 5109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:12.615458 5109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.615499 5109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:12.615527 5109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.621825 5109 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:32835
I20260807 06:42:12.621877 5284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:32835 every 8 connection(s)
I20260807 06:42:12.622264 5109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:42:12.625722 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 5109
I20260807 06:42:12.625870 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:42:12.626091 11723 external_mini_cluster.cc:1428] Setting key c5cef5c478a6544cc19809a00a7d5881
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:12.633139 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.623143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:52301 (local address 127.11.114.254:41031)
0807 06:42:12.623204 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.623206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.623446 (+ 240us) server_negotiation.cc:407] Connection header received
0807 06:42:12.623974 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.623976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.623988 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.624050 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:12.625029 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.625668 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.626983 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.627203 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.627284 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.629562 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:12.629564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:12.629564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:12.629572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:12.631431 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.632018 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.632019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.632020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.632080 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.632420 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.632421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.632421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.632543 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:12.632600 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.632995 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.633069 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 06:42:12.633500 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.623115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:41031 (local address 127.11.114.194:52301)
0807 06:42:12.623322 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.623359 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:12.623867 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:12.624126 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:12.624130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:12.624720 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:12.624924 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.624929 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.625766 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.625767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.626891 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.626893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.627293 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.627294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.627431 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.628008 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:12.628017 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:12.629437 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:12.631563 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:12.631572 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:12.631575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:12.631933 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:12.632160 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:12.632161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:12.632162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:12.632337 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:12.632622 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:12.632631 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:12.632829 (+ 198us) client_negotiation.cc:769] Sending connection context
0807 06:42:12.632974 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 06:42:12.633189 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260807 06:42:12.634141 5285 heartbeater.cc:347] Connected to a master server at 127.11.114.254:41031
I20260807 06:42:12.634277 5285 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:12.634482 5285 heartbeater.cc:511] Master 127.11.114.254:41031 requested a full tablet report, sending...
I20260807 06:42:12.634929 4843 ts_manager.cc:194] Registered new tserver with Master: d1cfc96ebc244cbb980850369180de8a (127.11.114.194:32835)
I20260807 06:42:12.635313 4843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:52301
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:42:12.657068 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:41031
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:12.745473 5294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:12.745754 5294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:12.745798 5294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:12.746974 5294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:12.747053 5294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:12.747109 5294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:42:12.748606 5294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:41031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:12.748893 5294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:12.749152 5294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:12.754459 5301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:12.754474 5304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:12.754500 5302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:12.754976 5294 server_base.cc:1034] running on GCE node
I20260807 06:42:12.755210 5294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:12.755463 5294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:12.756631 5294 hybrid_clock.cc:648] HybridClock initialized: now 1786084932756624 us; error 37 us; skew 500 ppm
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:12.759318 5294 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:42:12.759807 5294 webserver.cc:460] Webserver started at http://127.11.114.195:41433/ using document root <none> and password file <none>
I20260807 06:42:12.760076 5294 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:12.760154 5294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:12.760309 5294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:12.761364 5294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "9255322bedb24190bb2d0c545d8c89e3"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "adf6cc9538300b98a19ff67074e1d0e3"
server_key_iv: "712c19ad4a4af754be38e03b9811cebc"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.761750 5294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "9255322bedb24190bb2d0c545d8c89e3"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "adf6cc9538300b98a19ff67074e1d0e3"
server_key_iv: "712c19ad4a4af754be38e03b9811cebc"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.763252 5294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:12.764015 5314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.764237 5294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:12.764353 5294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "9255322bedb24190bb2d0c545d8c89e3"
format_stamp: "Formatted at 2026-08-07 06:42:12 on dist-test-slave-gkw7"
server_key: "adf6cc9538300b98a19ff67074e1d0e3"
server_key_iv: "712c19ad4a4af754be38e03b9811cebc"
server_key_version: "encryptionkey@0"
I20260807 06:42:12.764434 5294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:12.778355 5294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:12.780321 5294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:12.780485 5294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:12.780717 5294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:12.781081 5294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:12.781152 5294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.781195 5294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:12.781222 5294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:12.787926 5294 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:42345
I20260807 06:42:12.787971 5476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:42345 every 8 connection(s)
I20260807 06:42:12.788303 5294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:42:12.793654 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 5294
I20260807 06:42:12.793749 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:42:12.793938 11723 external_mini_cluster.cc:1428] Setting key 87dce6bf121a21b28bb5dc5a5ecbfac9
Aug 07 06:42:12 dist-test-slave-gkw7 krb5kdc[4780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084932, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:12.798095 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.789053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:47247 (local address 127.11.114.254:41031)
0807 06:42:12.789099 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.789101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.789343 (+ 242us) server_negotiation.cc:407] Connection header received
0807 06:42:12.789938 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.789939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.789952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.790011 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:12.791003 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.791524 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.792570 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.792780 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.792870 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.795579 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:12.795581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:12.795581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:12.795588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:12.796836 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.797212 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.797212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.797213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.797268 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.797486 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.797487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.797487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.797653 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:12.797708 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.797958 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.798027 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:12.798328 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.789031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:41031 (local address 127.11.114.195:47247)
0807 06:42:12.789208 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.789250 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:12.789831 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:12.790087 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:12.790092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:12.790697 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:12.790896 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.790902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.791626 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.791627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.792471 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.792473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.792880 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.792881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.793012 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.793615 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:12.793627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:12.795455 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:12.796913 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:12.796920 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:12.796922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:12.797143 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:12.797329 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:12.797329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:12.797329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:12.797430 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:12.797711 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:12.797716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:12.797842 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 06:42:12.797931 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:42:12.798055 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260807 06:42:12.798909 5478 heartbeater.cc:347] Connected to a master server at 127.11.114.254:41031
I20260807 06:42:12.799053 5478 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:12.799293 5478 heartbeater.cc:511] Master 127.11.114.254:41031 requested a full tablet report, sending...
I20260807 06:42:12.799779 4843 ts_manager.cc:194] Registered new tserver with Master: 9255322bedb24190bb2d0c545d8c89e3 (127.11.114.195:42345)
I20260807 06:42:12.800241 4843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:47247
I20260807 06:42:12.805956 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:42:12.813755 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.807323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57232 (local address 127.11.114.254:41031)
0807 06:42:12.807387 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.807389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.807499 (+ 110us) server_negotiation.cc:407] Connection header received
0807 06:42:12.807593 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.807594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.807604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.807683 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:12.808252 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.808748 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.809507 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.809742 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.809830 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.811202 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:12.811204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:12.811204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:12.811211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:12.812443 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.812849 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.812850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.812850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.812912 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:12.813142 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:12.813143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:12.813144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:12.813292 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:12.813410 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.813587 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.813651 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:42:12.815418 4843 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:42:12.816936 4843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:42:12.823747 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.819383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:34067 (local address 127.0.0.1:33016)
0807 06:42:12.819922 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.819931 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:12.819976 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:12.820401 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:12.820402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:12.820404 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:12.820473 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.820475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.821361 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.821362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.822705 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.822708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.823441 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.823443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.823523 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.823538 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:42:12.823606 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 06:42:12.823660 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":8,"thread_start_us":98,"threads_started":1}
I20260807 06:42:12.823858 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.819548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.11.114.195:42345)
0807 06:42:12.819753 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.819758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.819859 (+ 101us) server_negotiation.cc:407] Connection header received
0807 06:42:12.819956 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.819959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.819975 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.820103 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:12.820429 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.821673 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.822789 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.823532 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.823618 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.823686 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.823730 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.823770 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260807 06:42:12.824054 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.819496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.11.114.193:34067)
0807 06:42:12.820157 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.820162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.820173 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:42:12.820210 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.820212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.820226 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.820312 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:12.820556 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.821253 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.822820 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.823349 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.823443 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.823504 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.823926 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.823968 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":89,"threads_started":1}
I20260807 06:42:12.824012 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.819589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:32835 (local address 127.0.0.1:51244)
0807 06:42:12.819987 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.819995 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:12.820038 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:12.820407 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:12.820408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:12.820410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:12.820461 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.820462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.821730 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.821731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.822833 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.822835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.823757 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.823758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.823816 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.823821 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:12.823870 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:42:12.823918 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":5,"thread_start_us":79,"threads_started":1}
I20260807 06:42:12.824254 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.819722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51244 (local address 127.11.114.194:32835)
0807 06:42:12.819922 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.819927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.820170 (+ 243us) server_negotiation.cc:407] Connection header received
0807 06:42:12.820220 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.820223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.820241 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.820319 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:12.820542 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.821614 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.822911 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.823566 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.823671 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.823722 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.824093 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.824177 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":110,"threads_started":1}
I20260807 06:42:12.824070 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.819373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:42345 (local address 127.0.0.1:33316)
0807 06:42:12.819733 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.819779 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:12.819864 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:12.820261 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:12.820266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:12.820269 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:12.820343 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.820345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.821797 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.821798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.822702 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.822705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.823611 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.823612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.823664 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.823667 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:42:12.823702 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 06:42:12.823733 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":60,"threads_started":1}
I20260807 06:42:12.824919 5013 tablet_service.cc:1467] Processing CreateTablet for tablet 294561f6cb7d4ddd8dc5798d0d826b6a (DEFAULT_TABLE table=test-table [id=be63b652a67c4b479f41f05adda6ea4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:12.825133 5190 tablet_service.cc:1467] Processing CreateTablet for tablet 294561f6cb7d4ddd8dc5798d0d826b6a (DEFAULT_TABLE table=test-table [id=be63b652a67c4b479f41f05adda6ea4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:12.825337 5013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 294561f6cb7d4ddd8dc5798d0d826b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:12.825403 5190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 294561f6cb7d4ddd8dc5798d0d826b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:12.825321 5378 tablet_service.cc:1467] Processing CreateTablet for tablet 294561f6cb7d4ddd8dc5798d0d826b6a (DEFAULT_TABLE table=test-table [id=be63b652a67c4b479f41f05adda6ea4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:12.825655 5378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 294561f6cb7d4ddd8dc5798d0d826b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:12.828255 5502 tablet_bootstrap.cc:492] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: Bootstrap starting.
I20260807 06:42:12.829066 5504 tablet_bootstrap.cc:492] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7: Bootstrap starting.
I20260807 06:42:12.829102 5502 tablet_bootstrap.cc:654] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:12.829190 5503 tablet_bootstrap.cc:492] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: Bootstrap starting.
I20260807 06:42:12.829690 5502 log.cc:824] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:12.829866 5504 tablet_bootstrap.cc:654] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:12.829866 5503 tablet_bootstrap.cc:654] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:12.830488 5502 tablet_bootstrap.cc:492] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: No bootstrap required, opened a new log
I20260807 06:42:12.830497 5503 log.cc:824] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:12.830497 5504 log.cc:824] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:12.830574 5502 ts_tablet_manager.cc:1397] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:42:12.831456 5503 tablet_bootstrap.cc:492] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: No bootstrap required, opened a new log
I20260807 06:42:12.831507 5504 tablet_bootstrap.cc:492] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7: No bootstrap required, opened a new log
I20260807 06:42:12.831533 5503 ts_tablet_manager.cc:1397] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:42:12.831557 5504 ts_tablet_manager.cc:1397] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 06:42:12.832614 5502 raft_consensus.cc:348] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.832792 5502 raft_consensus.cc:374] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:12.832857 5502 raft_consensus.cc:729] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1cfc96ebc244cbb980850369180de8a, State: Initialized, Role: FOLLOWER
I20260807 06:42:12.832991 5502 consensus_queue.cc:260] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.833309 5285 heartbeater.cc:503] Master 127.11.114.254:41031 was elected leader, sending a full tablet report...
I20260807 06:42:12.833607 5502 ts_tablet_manager.cc:1428] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:42:12.833521 5503 raft_consensus.cc:348] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.833555 5504 raft_consensus.cc:348] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.833734 5504 raft_consensus.cc:374] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:12.833730 5503 raft_consensus.cc:374] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:12.833777 5503 raft_consensus.cc:729] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9255322bedb24190bb2d0c545d8c89e3, State: Initialized, Role: FOLLOWER
I20260807 06:42:12.833817 5504 raft_consensus.cc:729] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36281f21209a45f9a7a8a76903aa1bf7, State: Initialized, Role: FOLLOWER
I20260807 06:42:12.833928 5504 consensus_queue.cc:260] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.833923 5503 consensus_queue.cc:260] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.834218 5504 ts_tablet_manager.cc:1428] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 06:42:12.834223 5503 ts_tablet_manager.cc:1428] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:42:12.834257 5478 heartbeater.cc:503] Master 127.11.114.254:41031 was elected leader, sending a full tablet report...
I20260807 06:42:12.834324 5102 heartbeater.cc:503] Master 127.11.114.254:41031 was elected leader, sending a full tablet report...
I20260807 06:42:12.836134 5510 raft_consensus.cc:482] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:42:12.836222 5510 raft_consensus.cc:504] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.836578 5510 leader_election.cc:288] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9255322bedb24190bb2d0c545d8c89e3 (127.11.114.195:42345), 36281f21209a45f9a7a8a76903aa1bf7 (127.11.114.193:34067)
I20260807 06:42:12.839399 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.836922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:32957 (local address 127.11.114.193:34067)
0807 06:42:12.836985 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.836987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.836994 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:42:12.837013 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.837014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.837023 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.837095 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:12.837256 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.837775 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.838665 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.839041 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.839124 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.839133 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.839298 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.839333 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:42:12.839418 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.836826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:51863 (local address 127.11.114.195:42345)
0807 06:42:12.836916 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.836918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.836928 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:42:12.836951 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.836952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.836960 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.837011 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:12.837205 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.837752 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.838659 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.839024 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.839120 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.839129 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.839322 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.839357 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:42:12.839448 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.836692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:42345 (local address 127.11.114.194:51863)
0807 06:42:12.836729 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.836742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:12.836784 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:12.837029 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:12.837031 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:12.837033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:12.837089 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.837090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.837833 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.837834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.838547 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.838549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.839120 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.839121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.839234 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.839237 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:42:12.839313 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:42:12.839353 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":20}
I20260807 06:42:12.839447 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.836734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:34067 (local address 127.11.114.194:32957)
0807 06:42:12.836867 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.836875 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:12.836918 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:12.837098 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:12.837098 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:12.837100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:12.837173 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.837175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.837854 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.837855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.838574 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:12.838576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.839124 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.839125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:12.839222 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.839225 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:42:12.839291 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:42:12.839330 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260807 06:42:12.839722 5408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "294561f6cb7d4ddd8dc5798d0d826b6a" candidate_uuid: "d1cfc96ebc244cbb980850369180de8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9255322bedb24190bb2d0c545d8c89e3" is_pre_election: true
I20260807 06:42:12.839841 5408 raft_consensus.cc:2454] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1cfc96ebc244cbb980850369180de8a in term 0.
I20260807 06:42:12.839890 5037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "294561f6cb7d4ddd8dc5798d0d826b6a" candidate_uuid: "d1cfc96ebc244cbb980850369180de8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36281f21209a45f9a7a8a76903aa1bf7" is_pre_election: true
I20260807 06:42:12.840030 5037 raft_consensus.cc:2454] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1cfc96ebc244cbb980850369180de8a in term 0.
I20260807 06:42:12.840075 5134 leader_election.cc:302] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9255322bedb24190bb2d0c545d8c89e3, d1cfc96ebc244cbb980850369180de8a; no voters:
I20260807 06:42:12.840265 5510 raft_consensus.cc:2781] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:42:12.840307 5510 raft_consensus.cc:482] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:42:12.840322 5510 raft_consensus.cc:3037] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:12.840974 5510 raft_consensus.cc:504] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.841112 5510 leader_election.cc:288] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [CANDIDATE]: Term 1 election: Requested vote from peers 9255322bedb24190bb2d0c545d8c89e3 (127.11.114.195:42345), 36281f21209a45f9a7a8a76903aa1bf7 (127.11.114.193:34067)
I20260807 06:42:12.841257 5037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "294561f6cb7d4ddd8dc5798d0d826b6a" candidate_uuid: "d1cfc96ebc244cbb980850369180de8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36281f21209a45f9a7a8a76903aa1bf7"
I20260807 06:42:12.841321 5408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "294561f6cb7d4ddd8dc5798d0d826b6a" candidate_uuid: "d1cfc96ebc244cbb980850369180de8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9255322bedb24190bb2d0c545d8c89e3"
I20260807 06:42:12.841351 5037 raft_consensus.cc:3037] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:12.841399 5408 raft_consensus.cc:3037] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:12.842123 5408 raft_consensus.cc:2454] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1cfc96ebc244cbb980850369180de8a in term 1.
I20260807 06:42:12.842130 5037 raft_consensus.cc:2454] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1cfc96ebc244cbb980850369180de8a in term 1.
I20260807 06:42:12.842314 5133 leader_election.cc:302] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36281f21209a45f9a7a8a76903aa1bf7, d1cfc96ebc244cbb980850369180de8a; no voters:
I20260807 06:42:12.842406 5510 raft_consensus.cc:2781] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:12.842535 5510 raft_consensus.cc:686] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 1 LEADER]: Becoming Leader. State: Replica: d1cfc96ebc244cbb980850369180de8a, State: Running, Role: LEADER
I20260807 06:42:12.842638 5510 consensus_queue.cc:237] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } }
I20260807 06:42:12.843461 4837 catalog_manager.cc:5495] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a reported cstate change: term changed from 0 to 1, leader changed from <none> to d1cfc96ebc244cbb980850369180de8a (127.11.114.194). New cstate: current_term: 1 leader_uuid: "d1cfc96ebc244cbb980850369180de8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1cfc96ebc244cbb980850369180de8a" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 32835 } health_report { overall_health: HEALTHY } } }
I20260807 06:42:12.851847 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.849447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.11.114.194:32835)
0807 06:42:12.849514 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:12.849516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:12.849559 (+ 43us) server_negotiation.cc:407] Connection header received
0807 06:42:12.849592 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:12.849593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:12.849611 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:12.849681 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:12.849838 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.850289 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.851052 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:12.851259 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:12.851333 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:12.851505 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:12.851605 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:12.851713 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:12.851745 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 06:42:12.851755 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:42:12.853760 5408 raft_consensus.cc:1261] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 1 FOLLOWER]: Refusing update from remote peer d1cfc96ebc244cbb980850369180de8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:12.853773 5037 raft_consensus.cc:1261] T 294561f6cb7d4ddd8dc5798d0d826b6a P 36281f21209a45f9a7a8a76903aa1bf7 [term 1 FOLLOWER]: Refusing update from remote peer d1cfc96ebc244cbb980850369180de8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:12.854116 5517 consensus_queue.cc:1035] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "36281f21209a45f9a7a8a76903aa1bf7" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 34067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:12.854233 5510 consensus_queue.cc:1035] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9255322bedb24190bb2d0c545d8c89e3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 42345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:12.856626 5522 mvcc.cc:205] Tried to move back new op lower bound from 7315803884965679104 to 7315803884924186624. Current Snapshot: MvccSnapshot[applied={T|T < 7315803884965679104}]
I20260807 06:42:12.856825 5520 mvcc.cc:205] Tried to move back new op lower bound from 7315803884965679104 to 7315803884924186624. Current Snapshot: MvccSnapshot[applied={T|T < 7315803884965679104 or (T in {7315803884965679104})}]
I20260807 06:42:12.857053 5526 mvcc.cc:205] Tried to move back new op lower bound from 7315803884965679104 to 7315803884924186624. Current Snapshot: MvccSnapshot[applied={T|T < 7315803884965679104}]
I20260807 06:42:12.859087 4843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:12.859192 4843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:12.859901 4843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82002df171ca4ea59d75d00058457d54: Sending DeleteTablet for 3 replicas of tablet 294561f6cb7d4ddd8dc5798d0d826b6a
I20260807 06:42:12.860256 5013 tablet_service.cc:1514] Processing DeleteTablet for tablet 294561f6cb7d4ddd8dc5798d0d826b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:12 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:33016
I20260807 06:42:12.860313 5190 tablet_service.cc:1514] Processing DeleteTablet for tablet 294561f6cb7d4ddd8dc5798d0d826b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:12 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:51244
I20260807 06:42:12.860320 5378 tablet_service.cc:1514] Processing DeleteTablet for tablet 294561f6cb7d4ddd8dc5798d0d826b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:12 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:33316
I20260807 06:42:12.860534 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 4930
I20260807 06:42:12.862900 5535 tablet_replica.cc:330] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: stopping tablet replica
I20260807 06:42:12.862999 5535 raft_consensus.cc:2229] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:12.863097 5535 raft_consensus.cc:2258] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:12.863569 5535 ts_tablet_manager.cc:1905] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:12.865326 5535 ts_tablet_manager.cc:1918] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:12.865393 5535 log.cc:1196] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/294561f6cb7d4ddd8dc5798d0d826b6a
I20260807 06:42:12.865720 5535 ts_tablet_manager.cc:1939] T 294561f6cb7d4ddd8dc5798d0d826b6a P 9255322bedb24190bb2d0c545d8c89e3: Deleting consensus metadata
I20260807 06:42:12.865895 5538 tablet_replica.cc:330] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: stopping tablet replica
I20260807 06:42:12.865981 5538 raft_consensus.cc:2229] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:42:12.866160 4819 catalog_manager.cc:4841] TS 9255322bedb24190bb2d0c545d8c89e3 (127.11.114.195:42345): tablet 294561f6cb7d4ddd8dc5798d0d826b6a (table test-table [id=be63b652a67c4b479f41f05adda6ea4d]) successfully deleted
I20260807 06:42:12.866202 5538 raft_consensus.cc:2258] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:12.866578 5538 ts_tablet_manager.cc:1905] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:12.867997 5538 ts_tablet_manager.cc:1918] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:12.868089 5538 log.cc:1196] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/294561f6cb7d4ddd8dc5798d0d826b6a
I20260807 06:42:12.868427 5538 ts_tablet_manager.cc:1939] T 294561f6cb7d4ddd8dc5798d0d826b6a P d1cfc96ebc244cbb980850369180de8a: Deleting consensus metadata
I20260807 06:42:12.868827 4819 catalog_manager.cc:4841] TS d1cfc96ebc244cbb980850369180de8a (127.11.114.194:32835): tablet 294561f6cb7d4ddd8dc5798d0d826b6a (table test-table [id=be63b652a67c4b479f41f05adda6ea4d]) successfully deleted
W20260807 06:42:12.872018 4818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:34067 (error 108)
I20260807 06:42:12.872237 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 5109
I20260807 06:42:12.881656 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:12.881450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:34067 (local address 127.0.0.1:33024)
0807 06:42:12.881519 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:12.881618 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:34067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:42:12.881857 4818 catalog_manager.cc:4584] TS 36281f21209a45f9a7a8a76903aa1bf7 (127.11.114.193:34067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 294561f6cb7d4ddd8dc5798d0d826b6a: Network error: Client connection negotiation failed: client connection to 127.11.114.193:34067: connect: Connection refused (error 111)
I20260807 06:42:12.883152 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 5294
I20260807 06:42:12.893283 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 4797
2026-08-07T06:42:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2095 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T06:42:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:12Z Disabled control of system clock
I20260807 06:42:12.929692 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39479
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:13.015961 5547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:13.016199 5547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:13.016222 5547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:13.017432 5547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:13.017479 5547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:13.017493 5547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:13.017560 5547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:13.017587 5547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:13.018953 5547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:13.019222 5547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:13.019531 5547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:13.024580 5557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:13.024590 5554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:13.024606 5553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:13.024883 5547 server_base.cc:1034] running on GCE node
I20260807 06:42:13.025161 5547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:13.025490 5547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:13.026695 5547 hybrid_clock.cc:648] HybridClock initialized: now 1786084933026683 us; error 29 us; skew 500 ppm
I20260807 06:42:13.032742 5547 webserver.cc:460] Webserver started at http://127.11.114.254:37045/ using document root <none> and password file <none>
I20260807 06:42:13.033041 5547 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:13.033123 5547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:13.033317 5547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:13.034614 5547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "afaed1320cde42b7b62316abe647722e"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "92b62610d4a8f7a226c7ada3051d8f27"
server_key_iv: "5bea6e12ae8493c48112d36575499d58"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.035055 5547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "afaed1320cde42b7b62316abe647722e"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "92b62610d4a8f7a226c7ada3051d8f27"
server_key_iv: "5bea6e12ae8493c48112d36575499d58"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.036559 5547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:42:13.037365 5566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.037607 5547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:13.037732 5547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "afaed1320cde42b7b62316abe647722e"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "92b62610d4a8f7a226c7ada3051d8f27"
server_key_iv: "5bea6e12ae8493c48112d36575499d58"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.037825 5547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:13.055789 5547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:13.056181 5547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:13.056361 5547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:13.061976 5547 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39479
I20260807 06:42:13.062005 5634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39479 every 8 connection(s)
I20260807 06:42:13.062456 5547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:13.063498 5636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:13.064800 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 5547
I20260807 06:42:13.064949 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:13.065164 11723 external_mini_cluster.cc:1428] Setting key b89c0c3afe82dd880ced87892f37a50d
I20260807 06:42:13.066385 5636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e: Bootstrap starting.
I20260807 06:42:13.067144 5636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:13.067497 5636 log.cc:824] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:13.068293 5636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e: No bootstrap required, opened a new log
I20260807 06:42:13.070500 5636 raft_consensus.cc:348] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afaed1320cde42b7b62316abe647722e" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39479 } }
I20260807 06:42:13.070719 5636 raft_consensus.cc:374] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:13.070794 5636 raft_consensus.cc:729] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afaed1320cde42b7b62316abe647722e, State: Initialized, Role: FOLLOWER
I20260807 06:42:13.070945 5636 consensus_queue.cc:260] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afaed1320cde42b7b62316abe647722e" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39479 } }
I20260807 06:42:13.071039 5636 raft_consensus.cc:388] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:13.071079 5636 raft_consensus.cc:482] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:13.071125 5636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:13.071743 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.065799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.11.114.254:39479)
0807 06:42:13.066031 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.066041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.066075 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:42:13.066614 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.066623 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.066711 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.066895 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:13.067124 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.068455 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.069323 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.070040 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.070201 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.070972 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:13.070976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:13.070976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:13.071195 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:13.071246 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:13.071262 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:13.071335 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.071426 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.071512 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
I20260807 06:42:13.071867 5636 raft_consensus.cc:504] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afaed1320cde42b7b62316abe647722e" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39479 } }
I20260807 06:42:13.071971 5636 leader_election.cc:302] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afaed1320cde42b7b62316abe647722e; no voters:
I20260807 06:42:13.072144 5636 leader_election.cc:288] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:13.072189 5642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:13.072397 5642 raft_consensus.cc:686] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [term 1 LEADER]: Becoming Leader. State: Replica: afaed1320cde42b7b62316abe647722e, State: Running, Role: LEADER
I20260807 06:42:13.072503 5636 sys_catalog.cc:564] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:13.072511 5642 consensus_queue.cc:237] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afaed1320cde42b7b62316abe647722e" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39479 } }
I20260807 06:42:13.072896 5643 sys_catalog.cc:455] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afaed1320cde42b7b62316abe647722e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afaed1320cde42b7b62316abe647722e" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39479 } } }
I20260807 06:42:13.073011 5643 sys_catalog.cc:458] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:13.072894 5644 sys_catalog.cc:455] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [sys.catalog]: SysCatalogTable state changed. Reason: New leader afaed1320cde42b7b62316abe647722e. Latest consensus state: current_term: 1 leader_uuid: "afaed1320cde42b7b62316abe647722e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afaed1320cde42b7b62316abe647722e" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39479 } } }
I20260807 06:42:13.073568 5644 sys_catalog.cc:458] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:13.073691 5648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:13.074198 5648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:13.075739 5648 catalog_manager.cc:1347] Generated new cluster ID: a11c69cb7e744758a2ad31eac921c221
I20260807 06:42:13.075798 5648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:13.089520 5648 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:13.090140 5648 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:13.096472 5648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e: Generated new TSK 0
I20260807 06:42:13.096694 5648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:13.106686 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39479
--builtin_ntp_servers=127.11.114.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:13.195480 5666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:13.195729 5666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:13.195777 5666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:13.196944 5666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:13.197016 5666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:13.197054 5666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:42:13.198450 5666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:13.198732 5666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:13.199060 5666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:13.204277 5671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:13.204406 5673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:13.204262 5677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:13.204918 5666 server_base.cc:1034] running on GCE node
I20260807 06:42:13.205132 5666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:13.205385 5666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:13.206548 5666 hybrid_clock.cc:648] HybridClock initialized: now 1786084933206533 us; error 29 us; skew 500 ppm
I20260807 06:42:13.207814 5666 webserver.cc:460] Webserver started at http://127.11.114.193:36853/ using document root <none> and password file <none>
I20260807 06:42:13.208086 5666 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:13.208160 5666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:13.208323 5666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:13.209381 5666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "409d8f62eb5b4fb09b153fe197e02e27"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "2c094271e00dec3d1944907a04c47d52"
server_key_iv: "718ea5c2ab2933c076dfc6eb45dca2a1"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.209774 5666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "409d8f62eb5b4fb09b153fe197e02e27"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "2c094271e00dec3d1944907a04c47d52"
server_key_iv: "718ea5c2ab2933c076dfc6eb45dca2a1"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.211148 5666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:13.211951 5690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.212134 5666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:13.212255 5666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "409d8f62eb5b4fb09b153fe197e02e27"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "2c094271e00dec3d1944907a04c47d52"
server_key_iv: "718ea5c2ab2933c076dfc6eb45dca2a1"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.212332 5666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:13.228571 5666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:13.228904 5666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:13.229059 5666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:13.229303 5666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:13.229744 5666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:13.229809 5666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.229853 5666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:13.229879 5666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.235937 5666 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:46743
I20260807 06:42:13.235982 5839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:46743 every 8 connection(s)
I20260807 06:42:13.236325 5666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:42:13.241415 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.237071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:47003 (local address 127.11.114.254:39479)
0807 06:42:13.237153 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.237156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.237380 (+ 224us) server_negotiation.cc:407] Connection header received
0807 06:42:13.237862 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.237864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.237879 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.237930 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:13.238574 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.239069 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.240021 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.240200 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.240252 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.241152 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:13.241153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:13.241154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:13.241158 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:13.241171 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:13.241175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:13.241218 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.241333 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.241355 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 06:42:13.241716 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.237026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39479 (local address 127.11.114.193:47003)
0807 06:42:13.237248 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.237282 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:13.237777 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:13.237988 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:13.237992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:13.237999 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:13.238506 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.238511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.239150 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.239151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.239945 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.239946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.240284 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.240285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.240398 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.241019 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:13.241020 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:13.241049 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:13.241070 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:13.241229 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:13.241233 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:13.241235 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:42:13.241327 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:42:13.241410 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
I20260807 06:42:13.242365 5840 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39479
I20260807 06:42:13.242471 5840 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:13.242666 5840 heartbeater.cc:511] Master 127.11.114.254:39479 requested a full tablet report, sending...
I20260807 06:42:13.243052 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 5666
I20260807 06:42:13.243173 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:42:13.243300 5586 ts_manager.cc:194] Registered new tserver with Master: 409d8f62eb5b4fb09b153fe197e02e27 (127.11.114.193:46743)
I20260807 06:42:13.243430 11723 external_mini_cluster.cc:1428] Setting key 0623685bca27c617336eba502eee5778
I20260807 06:42:13.244220 5586 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.193:47003
I20260807 06:42:13.244710 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39479
--builtin_ntp_servers=127.11.114.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:13.331045 5844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:13.331264 5844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:13.331288 5844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:13.332350 5844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:13.332382 5844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:13.332406 5844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:42:13.333784 5844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:13.333983 5844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:13.334247 5844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:13.339324 5856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:13.339324 5851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:13.339517 5844 server_base.cc:1034] running on GCE node
W20260807 06:42:13.339403 5853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:13.339833 5844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:13.340080 5844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:13.341254 5844 hybrid_clock.cc:648] HybridClock initialized: now 1786084933341231 us; error 39 us; skew 500 ppm
I20260807 06:42:13.342638 5844 webserver.cc:460] Webserver started at http://127.11.114.194:33979/ using document root <none> and password file <none>
I20260807 06:42:13.342916 5844 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:13.342991 5844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:13.343166 5844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:13.344220 5844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "c986769bb8f04ca0b51fcb92c34434c3"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "c3bbbc8bb9ed5f89330898e34754bf40"
server_key_iv: "e32b55cbc8c2ca43205095349acef71b"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.344594 5844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "c986769bb8f04ca0b51fcb92c34434c3"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "c3bbbc8bb9ed5f89330898e34754bf40"
server_key_iv: "e32b55cbc8c2ca43205095349acef71b"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.345999 5844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:13.346742 5865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.346918 5844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:42:13.347026 5844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "c986769bb8f04ca0b51fcb92c34434c3"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "c3bbbc8bb9ed5f89330898e34754bf40"
server_key_iv: "e32b55cbc8c2ca43205095349acef71b"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.347105 5844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:13.366516 5844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:13.366858 5844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:13.367022 5844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:13.367265 5844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:13.367611 5844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:13.367676 5844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.367713 5844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:13.367741 5844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.373857 5844 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:37487
I20260807 06:42:13.373898 6029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:37487 every 8 connection(s)
I20260807 06:42:13.374274 5844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:42:13.379522 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.375084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:53185 (local address 127.11.114.254:39479)
0807 06:42:13.375159 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.375161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.375384 (+ 223us) server_negotiation.cc:407] Connection header received
0807 06:42:13.375853 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.375854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.375863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.375933 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:13.376659 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.377198 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.378172 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.378361 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.378419 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.379289 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:13.379290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:13.379290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:13.379295 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:13.379309 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:13.379313 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:13.379369 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.379439 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.379462 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:42:13.379765 6033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.375066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39479 (local address 127.11.114.194:53185)
0807 06:42:13.375245 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.375287 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:13.375769 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:13.375988 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:13.375991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:13.375999 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:13.376512 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.376516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.377289 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.377290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.378088 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.378090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.378437 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.378438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.378546 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.379139 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:13.379140 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:13.379168 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:13.379190 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:13.379372 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:13.379374 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:13.379376 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:42:13.379423 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 06:42:13.379503 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260807 06:42:13.380376 6031 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39479
I20260807 06:42:13.380465 6031 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:13.380623 6031 heartbeater.cc:511] Master 127.11.114.254:39479 requested a full tablet report, sending...
I20260807 06:42:13.380956 5586 ts_manager.cc:194] Registered new tserver with Master: c986769bb8f04ca0b51fcb92c34434c3 (127.11.114.194:37487)
I20260807 06:42:13.381347 5586 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.194:53185
I20260807 06:42:13.382961 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 5844
I20260807 06:42:13.383102 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:42:13.383319 11723 external_mini_cluster.cc:1428] Setting key e99196a193c775a31922b2c96d7e956a
I20260807 06:42:13.384260 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39479
--builtin_ntp_servers=127.11.114.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:13.467509 6035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:13.467744 6035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:13.467784 6035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:13.468956 6035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:13.468998 6035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:13.469025 6035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:42:13.470486 6035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:13.470745 6035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:13.471069 6035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:13.476699 6041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:13.476662 6042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:13.476655 6047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:13.476920 6035 server_base.cc:1034] running on GCE node
I20260807 06:42:13.477144 6035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:13.477451 6035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:13.478612 6035 hybrid_clock.cc:648] HybridClock initialized: now 1786084933478592 us; error 44 us; skew 500 ppm
I20260807 06:42:13.479943 6035 webserver.cc:460] Webserver started at http://127.11.114.195:39397/ using document root <none> and password file <none>
I20260807 06:42:13.480223 6035 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:13.480307 6035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:13.480463 6035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:13.481650 6035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "c121d27d03324968967a0397c6811077"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "538ef39e1ea1fde53f229388188eeed5"
server_key_iv: "5c42c7860bfea166b0bc280b580d4090"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.482045 6035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "c121d27d03324968967a0397c6811077"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "538ef39e1ea1fde53f229388188eeed5"
server_key_iv: "5c42c7860bfea166b0bc280b580d4090"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.483420 6035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:42:13.484203 6062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.484416 6035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:13.484526 6035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "c121d27d03324968967a0397c6811077"
format_stamp: "Formatted at 2026-08-07 06:42:13 on dist-test-slave-gkw7"
server_key: "538ef39e1ea1fde53f229388188eeed5"
server_key_iv: "5c42c7860bfea166b0bc280b580d4090"
server_key_version: "encryptionkey@0"
I20260807 06:42:13.484602 6035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:13.514948 6035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:13.515365 6035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:13.515538 6035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:13.515812 6035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:13.516192 6035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:13.516252 6035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.516295 6035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:13.516323 6035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:13.522475 6035 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:38153
I20260807 06:42:13.522528 6212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:38153 every 8 connection(s)
I20260807 06:42:13.522913 6035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:42:13.528607 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.523752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:40207 (local address 127.11.114.254:39479)
0807 06:42:13.523822 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.523824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.524080 (+ 256us) server_negotiation.cc:407] Connection header received
0807 06:42:13.524635 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.524636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.524649 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.524710 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:13.525453 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.526042 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.527044 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.527255 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.527334 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.528274 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:13.528276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:13.528276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:13.528281 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:13.528296 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:13.528300 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:13.528371 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.528473 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.528503 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:42:13.528837 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.523745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:39479 (local address 127.11.114.195:40207)
0807 06:42:13.523949 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.523985 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:13.524514 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:13.524783 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:13.524787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:13.524796 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:13.525347 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.525352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.526144 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.526145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.526947 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.526949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.527340 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.527342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.527477 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.528098 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:13.528098 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:13.528128 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:13.528150 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:13.528373 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:13.528379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:13.528383 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:13.528474 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 06:42:13.528580 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260807 06:42:13.529392 6213 heartbeater.cc:347] Connected to a master server at 127.11.114.254:39479
I20260807 06:42:13.529476 6213 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:13.529712 6213 heartbeater.cc:511] Master 127.11.114.254:39479 requested a full tablet report, sending...
I20260807 06:42:13.530104 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 6035
I20260807 06:42:13.530246 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:42:13.530230 5586 ts_manager.cc:194] Registered new tserver with Master: c121d27d03324968967a0397c6811077 (127.11.114.195:38153)
I20260807 06:42:13.530514 11723 external_mini_cluster.cc:1428] Setting key 79a4d9b4348bd7cf1508b9a232a4c4ff
I20260807 06:42:13.530768 5586 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.195:40207
I20260807 06:42:13.531598 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:42:13.536305 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.533210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.11.114.254:39479)
0807 06:42:13.533283 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.533285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.533415 (+ 130us) server_negotiation.cc:407] Connection header received
0807 06:42:13.533469 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.533470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.533479 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.533619 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:13.533747 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.534255 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.534997 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.535221 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.535292 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.536042 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:13.536044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:13.536044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:13.536051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:13.536069 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:13.536074 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:13.536171 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.536231 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.536243 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:42:13.538127 5585 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:42:13.539711 5585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:42:13.546272 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.542010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38153 (local address 127.0.0.1:48574)
0807 06:42:13.542585 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.542594 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:13.542647 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:13.542902 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:13.542903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:13.542906 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:13.542954 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.542956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.543818 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.543819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.545152 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.545155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.545985 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.545986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.546064 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.546081 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 06:42:13.546157 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:42:13.546215 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":34,"thread_start_us":57,"threads_started":1}
I20260807 06:42:13.546638 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.542240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.11.114.195:38153)
0807 06:42:13.542445 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.542449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.542704 (+ 255us) server_negotiation.cc:407] Connection header received
0807 06:42:13.542748 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.542750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.542765 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.542837 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:13.543042 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.543723 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.545235 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.545887 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.545958 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.546045 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.546492 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.546544 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260807 06:42:13.546797 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.542008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:46743 (local address 127.0.0.1:59376)
0807 06:42:13.542431 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.542443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:13.542501 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:13.542810 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:13.542811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:13.542813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:13.542850 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.542852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.544114 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.544115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.545163 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.545164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.546541 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.546542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.546607 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.546611 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:13.546667 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:42:13.546707 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"spinlock_wait_cycles":768,"thread_start_us":93,"threads_started":1}
I20260807 06:42:13.547418 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.542003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:37487 (local address 127.0.0.1:49266)
0807 06:42:13.542171 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.542211 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:13.542277 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:13.542666 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:13.542669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:13.542672 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:13.542760 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.542763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.545259 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.545260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.546047 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.546049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.547152 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.547153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.547236 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.547241 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:13.547300 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:42:13.547344 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
I20260807 06:42:13.547389 6133 tablet_service.cc:1467] Processing CreateTablet for tablet 2cac60921488435ea8ee5d9d1dfe86db (DEFAULT_TABLE table=test-table [id=4ebbb5748a654aa2a381950fef0c9e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:13.547569 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.542090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.11.114.194:37487)
0807 06:42:13.542474 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.542479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.542489 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:42:13.542532 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.542534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.542553 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.542645 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:13.542858 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.544030 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.546134 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.546915 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.546983 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.547109 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.547394 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.547467 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":117,"threads_started":1}
I20260807 06:42:13.547832 6133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cac60921488435ea8ee5d9d1dfe86db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:13.548252 5938 tablet_service.cc:1467] Processing CreateTablet for tablet 2cac60921488435ea8ee5d9d1dfe86db (DEFAULT_TABLE table=test-table [id=4ebbb5748a654aa2a381950fef0c9e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:13.548616 5938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cac60921488435ea8ee5d9d1dfe86db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:13.549950 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.542268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.11.114.193:46743)
0807 06:42:13.542516 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.542519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.542527 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:42:13.542561 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.542562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.542574 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.542735 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:13.542928 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.544021 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.545611 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.546348 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.546420 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.546499 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.546703 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.549850 (+ 3147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":81,"threads_started":1}
I20260807 06:42:13.550701 5752 tablet_service.cc:1467] Processing CreateTablet for tablet 2cac60921488435ea8ee5d9d1dfe86db (DEFAULT_TABLE table=test-table [id=4ebbb5748a654aa2a381950fef0c9e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:13.551060 5752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cac60921488435ea8ee5d9d1dfe86db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:13.551492 6236 tablet_bootstrap.cc:492] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: Bootstrap starting.
I20260807 06:42:13.552122 6237 tablet_bootstrap.cc:492] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Bootstrap starting.
I20260807 06:42:13.552425 6236 tablet_bootstrap.cc:654] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:13.552749 6237 tablet_bootstrap.cc:654] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:13.552778 6236 log.cc:824] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:13.553094 6237 log.cc:824] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:13.553622 6236 tablet_bootstrap.cc:492] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: No bootstrap required, opened a new log
I20260807 06:42:13.553764 6236 ts_tablet_manager.cc:1397] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:42:13.553898 6237 tablet_bootstrap.cc:492] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: No bootstrap required, opened a new log
I20260807 06:42:13.554003 6237 ts_tablet_manager.cc:1397] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:42:13.554781 6241 tablet_bootstrap.cc:492] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27: Bootstrap starting.
I20260807 06:42:13.555444 6241 tablet_bootstrap.cc:654] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:13.555787 6241 log.cc:824] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:13.556495 6236 raft_consensus.cc:348] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.556617 6241 tablet_bootstrap.cc:492] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27: No bootstrap required, opened a new log
I20260807 06:42:13.556653 6236 raft_consensus.cc:374] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:13.556682 6241 ts_tablet_manager.cc:1397] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:42:13.556565 6237 raft_consensus.cc:348] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.556696 6236 raft_consensus.cc:729] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c121d27d03324968967a0397c6811077, State: Initialized, Role: FOLLOWER
I20260807 06:42:13.556720 6237 raft_consensus.cc:374] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:13.556762 6237 raft_consensus.cc:729] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c986769bb8f04ca0b51fcb92c34434c3, State: Initialized, Role: FOLLOWER
I20260807 06:42:13.556836 6236 consensus_queue.cc:260] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.556918 6237 consensus_queue.cc:260] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.557132 6236 ts_tablet_manager.cc:1428] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:13.557231 6237 ts_tablet_manager.cc:1428] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:13.557263 6213 heartbeater.cc:503] Master 127.11.114.254:39479 was elected leader, sending a full tablet report...
I20260807 06:42:13.557698 6031 heartbeater.cc:503] Master 127.11.114.254:39479 was elected leader, sending a full tablet report...
I20260807 06:42:13.559319 6241 raft_consensus.cc:348] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.559440 6241 raft_consensus.cc:374] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:13.559463 6241 raft_consensus.cc:729] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409d8f62eb5b4fb09b153fe197e02e27, State: Initialized, Role: FOLLOWER
I20260807 06:42:13.559589 6241 consensus_queue.cc:260] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.559868 6241 ts_tablet_manager.cc:1428] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:13.559906 5840 heartbeater.cc:503] Master 127.11.114.254:39479 was elected leader, sending a full tablet report...
W20260807 06:42:13.624868 6032 tablet.cc:2369] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:42:13.686205 6245 raft_consensus.cc:482] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:42:13.686350 6245 raft_consensus.cc:504] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.687136 6245 leader_election.cc:288] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c986769bb8f04ca0b51fcb92c34434c3 (127.11.114.194:37487), c121d27d03324968967a0397c6811077 (127.11.114.195:38153)
I20260807 06:42:13.691045 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.687233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:57695 (local address 127.11.114.194:37487)
0807 06:42:13.687306 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.687309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.687321 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:42:13.687356 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.687357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.687373 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.687459 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:13.687667 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.688570 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.690074 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.690649 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.690733 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.690742 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.690929 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.690973 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:42:13.691045 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.687125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:52057 (local address 127.11.114.195:38153)
0807 06:42:13.687186 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.687188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.687379 (+ 191us) server_negotiation.cc:407] Connection header received
0807 06:42:13.687711 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.687713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.687727 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.687795 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:13.687962 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.688733 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.690079 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.690649 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.690733 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.690742 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.690929 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.690979 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:42:13.691045 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.686960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:37487 (local address 127.11.114.193:57695)
0807 06:42:13.687017 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.687029 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:13.687073 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:13.687466 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:13.687467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:13.687469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:13.687570 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.687572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.688661 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.688662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.689938 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.689941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.690748 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.690749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.690834 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.690839 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:13.690904 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:42:13.690951 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":10}
I20260807 06:42:13.691051 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.686968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:38153 (local address 127.11.114.193:52057)
0807 06:42:13.687305 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.687314 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:13.687378 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:13.687806 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:13.687807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:13.687808 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:13.687873 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.687874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.688839 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.688840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.689994 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:13.689996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.690752 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.690753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:13.690845 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.690848 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:42:13.690904 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:42:13.690949 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":15,"thread_start_us":122,"threads_started":1}
I20260807 06:42:13.691470 5969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cac60921488435ea8ee5d9d1dfe86db" candidate_uuid: "409d8f62eb5b4fb09b153fe197e02e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c986769bb8f04ca0b51fcb92c34434c3" is_pre_election: true
I20260807 06:42:13.691583 6156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cac60921488435ea8ee5d9d1dfe86db" candidate_uuid: "409d8f62eb5b4fb09b153fe197e02e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c121d27d03324968967a0397c6811077" is_pre_election: true
I20260807 06:42:13.691641 5969 raft_consensus.cc:2454] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 409d8f62eb5b4fb09b153fe197e02e27 in term 0.
I20260807 06:42:13.691744 6156 raft_consensus.cc:2454] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 409d8f62eb5b4fb09b153fe197e02e27 in term 0.
I20260807 06:42:13.691884 5692 leader_election.cc:302] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 409d8f62eb5b4fb09b153fe197e02e27, c986769bb8f04ca0b51fcb92c34434c3; no voters:
I20260807 06:42:13.692032 6245 raft_consensus.cc:2781] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:42:13.692126 6245 raft_consensus.cc:482] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:42:13.692162 6245 raft_consensus.cc:3037] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:13.692930 6245 raft_consensus.cc:504] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.693095 6245 leader_election.cc:288] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [CANDIDATE]: Term 1 election: Requested vote from peers c986769bb8f04ca0b51fcb92c34434c3 (127.11.114.194:37487), c121d27d03324968967a0397c6811077 (127.11.114.195:38153)
I20260807 06:42:13.693271 5969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cac60921488435ea8ee5d9d1dfe86db" candidate_uuid: "409d8f62eb5b4fb09b153fe197e02e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c986769bb8f04ca0b51fcb92c34434c3"
I20260807 06:42:13.693292 6156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cac60921488435ea8ee5d9d1dfe86db" candidate_uuid: "409d8f62eb5b4fb09b153fe197e02e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c121d27d03324968967a0397c6811077"
I20260807 06:42:13.693349 5969 raft_consensus.cc:3037] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:13.693377 6156 raft_consensus.cc:3037] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:13.694103 5969 raft_consensus.cc:2454] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 409d8f62eb5b4fb09b153fe197e02e27 in term 1.
I20260807 06:42:13.694229 6156 raft_consensus.cc:2454] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 409d8f62eb5b4fb09b153fe197e02e27 in term 1.
I20260807 06:42:13.694275 5692 leader_election.cc:302] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 409d8f62eb5b4fb09b153fe197e02e27, c986769bb8f04ca0b51fcb92c34434c3; no voters:
I20260807 06:42:13.694368 6245 raft_consensus.cc:2781] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:13.694526 6245 raft_consensus.cc:686] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 1 LEADER]: Becoming Leader. State: Replica: 409d8f62eb5b4fb09b153fe197e02e27, State: Running, Role: LEADER
I20260807 06:42:13.694644 6245 consensus_queue.cc:237] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } }
I20260807 06:42:13.695540 5585 catalog_manager.cc:5495] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 409d8f62eb5b4fb09b153fe197e02e27 (127.11.114.193). New cstate: current_term: 1 leader_uuid: "409d8f62eb5b4fb09b153fe197e02e27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409d8f62eb5b4fb09b153fe197e02e27" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 46743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 } health_report { overall_health: UNKNOWN } } }
I20260807 06:42:13.727519 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.725287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.11.114.193:46743)
0807 06:42:13.725341 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:13.725342 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:13.725370 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:42:13.725407 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:13.725407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:13.725427 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:13.725484 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:13.725658 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.726123 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.726859 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:13.727040 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:13.727090 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:13.727217 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:13.727311 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:13.727383 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:13.727428 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 06:42:13.727438 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:42:13.729410 6156 raft_consensus.cc:1261] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 1 FOLLOWER]: Refusing update from remote peer 409d8f62eb5b4fb09b153fe197e02e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:13.729439 5969 raft_consensus.cc:1261] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 1 FOLLOWER]: Refusing update from remote peer 409d8f62eb5b4fb09b153fe197e02e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:13.729857 6250 consensus_queue.cc:1035] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c121d27d03324968967a0397c6811077" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 38153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:13.729996 6245 consensus_queue.cc:1035] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c986769bb8f04ca0b51fcb92c34434c3" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 37487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:13.733192 6256 mvcc.cc:205] Tried to move back new op lower bound from 7315803888552632320 to 7315803888413896704. Current Snapshot: MvccSnapshot[applied={T|T < 7315803888552632320}]
I20260807 06:42:13.734752 5585 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:13.734863 5585 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:13.735682 5585 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afaed1320cde42b7b62316abe647722e: Sending DeleteTablet for 3 replicas of tablet 2cac60921488435ea8ee5d9d1dfe86db
I20260807 06:42:13.736049 5752 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cac60921488435ea8ee5d9d1dfe86db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:13 UTC) from {username='slave'} at 127.0.0.1:59376
I20260807 06:42:13.736097 6133 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cac60921488435ea8ee5d9d1dfe86db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:13 UTC) from {username='slave'} at 127.0.0.1:48574
I20260807 06:42:13.736271 6267 tablet_replica.cc:330] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27: stopping tablet replica
I20260807 06:42:13.736307 5938 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cac60921488435ea8ee5d9d1dfe86db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:13 UTC) from {username='slave'} at 127.0.0.1:49266
I20260807 06:42:13.736377 6267 raft_consensus.cc:2229] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:42:13.736374 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 5666
I20260807 06:42:13.736474 6267 raft_consensus.cc:2258] T 2cac60921488435ea8ee5d9d1dfe86db P 409d8f62eb5b4fb09b153fe197e02e27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:13.742841 6271 tablet_replica.cc:330] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: stopping tablet replica
I20260807 06:42:13.742926 6271 raft_consensus.cc:2229] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:13.743000 6271 raft_consensus.cc:2258] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:13.743227 6268 tablet_replica.cc:330] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: stopping tablet replica
I20260807 06:42:13.743330 6268 raft_consensus.cc:2229] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:13.743413 6268 raft_consensus.cc:2258] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:13.743733 6271 ts_tablet_manager.cc:1905] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:13.745249 6268 ts_tablet_manager.cc:1905] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:13.745287 6271 ts_tablet_manager.cc:1918] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:13.745353 6271 log.cc:1196] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/2cac60921488435ea8ee5d9d1dfe86db
I20260807 06:42:13.745664 6271 ts_tablet_manager.cc:1939] T 2cac60921488435ea8ee5d9d1dfe86db P c986769bb8f04ca0b51fcb92c34434c3: Deleting consensus metadata
I20260807 06:42:13.746129 5569 catalog_manager.cc:4841] TS c986769bb8f04ca0b51fcb92c34434c3 (127.11.114.194:37487): tablet 2cac60921488435ea8ee5d9d1dfe86db (table test-table [id=4ebbb5748a654aa2a381950fef0c9e15]) successfully deleted
I20260807 06:42:13.746783 6268 ts_tablet_manager.cc:1918] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:13.746872 6268 log.cc:1196] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/2cac60921488435ea8ee5d9d1dfe86db
I20260807 06:42:13.747099 6268 ts_tablet_manager.cc:1939] T 2cac60921488435ea8ee5d9d1dfe86db P c121d27d03324968967a0397c6811077: Deleting consensus metadata
I20260807 06:42:13.747494 5570 catalog_manager.cc:4841] TS c121d27d03324968967a0397c6811077 (127.11.114.195:38153): tablet 2cac60921488435ea8ee5d9d1dfe86db (table test-table [id=4ebbb5748a654aa2a381950fef0c9e15]) successfully deleted
W20260807 06:42:13.749058 5568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:46743 (error 108)
I20260807 06:42:13.749305 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 5844
I20260807 06:42:13.749950 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:13.749507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:46743 (local address 127.0.0.1:59396)
0807 06:42:13.749849 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:13.749906 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:46743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314}
W20260807 06:42:13.753211 5568 catalog_manager.cc:4584] TS 409d8f62eb5b4fb09b153fe197e02e27 (127.11.114.193:46743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2cac60921488435ea8ee5d9d1dfe86db: Network error: Client connection negotiation failed: client connection to 127.11.114.193:46743: connect: Connection refused (error 111)
I20260807 06:42:13.758080 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 6035
I20260807 06:42:13.769901 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 5547
2026-08-07T06:42:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (872 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:13 dist-test-slave-gkw7 krb5kdc[6275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:13 dist-test-slave-gkw7 krb5kdc[6275](info): set up 2 sockets
Aug 07 06:42:13 dist-test-slave-gkw7 krb5kdc[6275](info): commencing operation
krb5kdc: starting...
W20260807 06:42:14.981228 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:15.111974 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:46371
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:46371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:15.197234 6294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:15.197441 6294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:15.197465 6294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:15.198659 6294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:15.198709 6294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:15.198726 6294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:15.198786 6294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:15.198815 6294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:15.200363 6294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:46371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:46371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:15.200629 6294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:15.200888 6294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:42:15.205989 6294 server_base.cc:1034] running on GCE node
W20260807 06:42:15.205957 6302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:15.205928 6308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:15.206032 6303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:15.206493 6294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:15.206812 6294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:15.207988 6294 hybrid_clock.cc:648] HybridClock initialized: now 1786084935207974 us; error 49 us; skew 500 ppm
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:15.210682 6294 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:15.211125 6294 webserver.cc:460] Webserver started at http://127.11.114.254:39947/ using document root <none> and password file <none>
I20260807 06:42:15.211374 6294 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:15.211443 6294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:15.211592 6294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:15.212637 6294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "ae6333d2bf9548be959a4087cab0e541"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "b6572901f5fc5cc431e3a9143064927c"
server_key_iv: "a8f491dc3d0c2d70295515206339f9b1"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.213001 6294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "ae6333d2bf9548be959a4087cab0e541"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "b6572901f5fc5cc431e3a9143064927c"
server_key_iv: "a8f491dc3d0c2d70295515206339f9b1"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.214468 6294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:15.215209 6317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.215382 6294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.215490 6294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "ae6333d2bf9548be959a4087cab0e541"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "b6572901f5fc5cc431e3a9143064927c"
server_key_iv: "a8f491dc3d0c2d70295515206339f9b1"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.215564 6294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:15.223512 6294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:15.225649 6294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:15.225828 6294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:15.229743 6294 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:46371
I20260807 06:42:15.229799 6392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:46371 every 8 connection(s)
I20260807 06:42:15.230180 6294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:15.231036 6393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:15.233278 6393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541: Bootstrap starting.
I20260807 06:42:15.233918 6393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:15.234206 6393 log.cc:824] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:15.234895 6393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541: No bootstrap required, opened a new log
I20260807 06:42:15.236449 6393 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6333d2bf9548be959a4087cab0e541" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46371 } }
I20260807 06:42:15.236579 6393 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:15.236632 6393 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae6333d2bf9548be959a4087cab0e541, State: Initialized, Role: FOLLOWER
I20260807 06:42:15.236804 6393 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6333d2bf9548be959a4087cab0e541" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46371 } }
I20260807 06:42:15.236867 6393 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:15.236888 6393 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:15.236954 6393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:15.237488 6393 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6333d2bf9548be959a4087cab0e541" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46371 } }
I20260807 06:42:15.237635 6393 leader_election.cc:302] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae6333d2bf9548be959a4087cab0e541; no voters:
I20260807 06:42:15.237788 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 6294
I20260807 06:42:15.237823 6393 leader_election.cc:288] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:15.237846 6396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:15.237926 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:15.237970 6396 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [term 1 LEADER]: Becoming Leader. State: Replica: ae6333d2bf9548be959a4087cab0e541, State: Running, Role: LEADER
I20260807 06:42:15.238082 6393 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:15.238051 6396 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6333d2bf9548be959a4087cab0e541" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46371 } }
I20260807 06:42:15.238155 11723 external_mini_cluster.cc:1428] Setting key 9c7d032bdfd676ee1bc9833e1a4eb856
I20260807 06:42:15.238391 6396 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae6333d2bf9548be959a4087cab0e541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6333d2bf9548be959a4087cab0e541" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46371 } } }
I20260807 06:42:15.238476 6396 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:15.238592 6401 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae6333d2bf9548be959a4087cab0e541. Latest consensus state: current_term: 1 leader_uuid: "ae6333d2bf9548be959a4087cab0e541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6333d2bf9548be959a4087cab0e541" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46371 } } }
I20260807 06:42:15.238690 6401 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:15.239336 6406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:15.239895 6406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:15.241935 6406 catalog_manager.cc:1347] Generated new cluster ID: 54d0924d04914c719861325333989ce9
I20260807 06:42:15.242014 6406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:15.247717 6406 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:15.248406 6406 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:15.250415 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.238858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.11.114.254:46371)
0807 06:42:15.239131 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:15.239141 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:15.239180 (+ 39us) server_negotiation.cc:407] Connection header received
0807 06:42:15.239311 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:15.239321 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:15.239382 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:15.240430 (+ 1048us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:15.240527 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.241920 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.243060 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.243694 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.243809 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.246691 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:15.246696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:15.246696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:15.246708 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:15.248712 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.249131 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.249134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.249135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.249240 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.249464 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.249465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.249465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.249748 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:15.249948 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:15.250025 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 06:42:15.250179 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
I20260807 06:42:15.252018 6406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541: Generated new TSK 0
I20260807 06:42:15.252225 6406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.114.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
Entry for principal kudu/127.11.114.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab.
I20260807 06:42:15.279819 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46371
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:15.365901 6434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:15.366125 6434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:15.366153 6434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:15.367300 6434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:15.367348 6434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:15.367377 6434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:42:15.368885 6434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.193.keytab
--principal=kudu/127.11.114.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:15.369133 6434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:15.369400 6434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:15.374866 6446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:15.374866 6444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:15.374888 6443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:15.375147 6434 server_base.cc:1034] running on GCE node
I20260807 06:42:15.375344 6434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:15.375595 6434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:15.376787 6434 hybrid_clock.cc:648] HybridClock initialized: now 1786084935376773 us; error 30 us; skew 500 ppm
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:15.379535 6434 init.cc:377] Logged in from keytab as kudu/127.11.114.193@KRBTEST.COM (short username kudu)
I20260807 06:42:15.379956 6434 webserver.cc:460] Webserver started at http://127.11.114.193:42063/ using document root <none> and password file <none>
I20260807 06:42:15.380169 6434 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:15.380213 6434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:15.380306 6434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:15.381310 6434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "620ccf02ad7f4a79940f51dfaf18152e"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "3be37b88881985ad8196a864e014855b"
server_key_iv: "def6733587a7732cb77cc7fb91ba949d"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.381688 6434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "620ccf02ad7f4a79940f51dfaf18152e"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "3be37b88881985ad8196a864e014855b"
server_key_iv: "def6733587a7732cb77cc7fb91ba949d"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.383002 6434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:15.383661 6457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.383833 6434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:42:15.383920 6434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "620ccf02ad7f4a79940f51dfaf18152e"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "3be37b88881985ad8196a864e014855b"
server_key_iv: "def6733587a7732cb77cc7fb91ba949d"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.383972 6434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:15.405761 6434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:15.408083 6434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:15.408193 6434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:15.408397 6434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:15.408660 6434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:15.408690 6434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.408713 6434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:15.408726 6434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.414764 6434 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:39517
I20260807 06:42:15.414829 6624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:39517 every 8 connection(s)
I20260807 06:42:15.415138 6434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:42:15.416144 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 6434
I20260807 06:42:15.416271 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:42:15.416515 11723 external_mini_cluster.cc:1428] Setting key 11c951a2a233af87abbc824eca3eaf71
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.193@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.194@KRBTEST.COM; defaulting to no policy
I20260807 06:42:15.425974 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.415924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:57451 (local address 127.11.114.254:46371)
0807 06:42:15.416005 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:15.416008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:15.416223 (+ 215us) server_negotiation.cc:407] Connection header received
0807 06:42:15.417146 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:15.417148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:15.417156 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:15.417240 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:15.418700 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.419438 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.420445 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.420670 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.420762 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.423000 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:15.423002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:15.423003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:15.423011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:15.424482 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.424913 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.424915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.424915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.424977 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.425233 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.425234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.425235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.425383 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:15.425458 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:15.425790 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 06:42:15.425878 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:42:15.426167 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.415875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46371 (local address 127.11.114.193:57451)
0807 06:42:15.416067 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:15.416119 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:15.416854 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:15.417372 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:15.417376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:15.418300 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:15.418588 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.418594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.419522 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.419524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.420349 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.420351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.420772 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.420774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.420910 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.421434 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:15.421442 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:15.422902 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:15.424583 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:15.424590 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:15.424592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:15.424818 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:15.425052 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:15.425053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:15.425053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:15.425163 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:15.425461 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:15.425468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:15.425654 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 06:42:15.425766 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:42:15.425923 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":111,"threads_started":1}
I20260807 06:42:15.426784 6625 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46371
I20260807 06:42:15.426896 6625 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:15.427102 6625 heartbeater.cc:511] Master 127.11.114.254:46371 requested a full tablet report, sending...
I20260807 06:42:15.427621 6340 ts_manager.cc:194] Registered new tserver with Master: 620ccf02ad7f4a79940f51dfaf18152e (127.11.114.193:39517)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.194@KRBTEST.COM" created.
I20260807 06:42:15.428381 6340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.193@KRBTEST.COM'} at 127.11.114.193:57451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
Entry for principal kudu/127.11.114.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab.
I20260807 06:42:15.444715 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46371
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:15.527343 6637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:15.527552 6637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:15.527575 6637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:15.528647 6637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:15.528682 6637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:15.528707 6637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:42:15.530128 6637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.194.keytab
--principal=kudu/127.11.114.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:15.530332 6637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:15.530508 6637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:42:15.535797 6637 server_base.cc:1034] running on GCE node
W20260807 06:42:15.535763 6646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:15.535784 6643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:15.535750 6644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:15.536211 6637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:15.536466 6637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:15.537621 6637 hybrid_clock.cc:648] HybridClock initialized: now 1786084935537609 us; error 40 us; skew 500 ppm
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:15.540369 6637 init.cc:377] Logged in from keytab as kudu/127.11.114.194@KRBTEST.COM (short username kudu)
I20260807 06:42:15.540766 6637 webserver.cc:460] Webserver started at http://127.11.114.194:41839/ using document root <none> and password file <none>
I20260807 06:42:15.540964 6637 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:15.541004 6637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:15.541097 6637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:15.542168 6637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "f1fa53d913aa4084a3dcbb03a10da892"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "9906e1687aa343d0af395be289a508fa"
server_key_iv: "5aa1676d63f7dd136e1458aff20e606c"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.542490 6637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "f1fa53d913aa4084a3dcbb03a10da892"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "9906e1687aa343d0af395be289a508fa"
server_key_iv: "5aa1676d63f7dd136e1458aff20e606c"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.543767 6637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:42:15.544446 6658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.544654 6637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:42:15.544755 6637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "f1fa53d913aa4084a3dcbb03a10da892"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "9906e1687aa343d0af395be289a508fa"
server_key_iv: "5aa1676d63f7dd136e1458aff20e606c"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.544812 6637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:15.556428 6637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:15.558569 6637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:15.558709 6637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:15.558911 6637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:15.559267 6637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:15.559305 6637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.559357 6637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:15.559388 6637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.566365 6637 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:44787
I20260807 06:42:15.566406 6819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:44787 every 8 connection(s)
I20260807 06:42:15.566792 6637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:42:15.570147 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 6637
I20260807 06:42:15.570297 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:42:15.570509 11723 external_mini_cluster.cc:1428] Setting key b32ccb42508969fa851371c8a38f22d0
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.194@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:15.577039 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.567662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:39225 (local address 127.11.114.254:46371)
0807 06:42:15.567727 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:15.567729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:15.567909 (+ 180us) server_negotiation.cc:407] Connection header received
0807 06:42:15.568454 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:15.568456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:15.568469 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:15.568525 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:15.569509 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.570075 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.571386 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.571584 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.571662 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.573926 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:15.573928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:15.573928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:15.573937 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:15.575451 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.575932 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.575933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.575934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.576006 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.576299 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.576300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.576300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.576440 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:15.576524 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:15.576848 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 06:42:15.576938 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:42:15.577250 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.567619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46371 (local address 127.11.114.194:39225)
0807 06:42:15.567779 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:15.567828 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:15.568339 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:15.568607 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:15.568613 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:15.569201 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:15.569401 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.569406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.570174 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.570176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.571297 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.571301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.571668 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.571669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.571800 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.572329 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:15.572338 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:15.573831 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:15.575576 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:15.575583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:15.575586 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:15.575841 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:15.576084 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:15.576085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:15.576085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:15.576213 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:15.576513 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:15.576520 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:15.576693 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 06:42:15.576810 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 06:42:15.576986 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260807 06:42:15.577826 6820 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46371
I20260807 06:42:15.577931 6820 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:15.578151 6820 heartbeater.cc:511] Master 127.11.114.254:46371 requested a full tablet report, sending...
I20260807 06:42:15.578611 6340 ts_manager.cc:194] Registered new tserver with Master: f1fa53d913aa4084a3dcbb03a10da892 (127.11.114.194:44787)
I20260807 06:42:15.579175 6340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.194@KRBTEST.COM'} at 127.11.114.194:39225
WARNING: no policy specified for kudu/127.11.114.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
Entry for principal kudu/127.11.114.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab.
I20260807 06:42:15.598363 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46371
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:15.685881 6827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:15.686102 6827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:15.686127 6827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:15.687394 6827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:15.687471 6827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:15.687510 6827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:42:15.688951 6827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.195.keytab
--principal=kudu/127.11.114.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:46371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:15.689150 6827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:15.689334 6827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:15.694537 6835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:15.694566 6837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:15.694566 6834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:15.694844 6827 server_base.cc:1034] running on GCE node
I20260807 06:42:15.695125 6827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:15.695351 6827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:15.696534 6827 hybrid_clock.cc:648] HybridClock initialized: now 1786084935696505 us; error 45 us; skew 500 ppm
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:15.699455 6827 init.cc:377] Logged in from keytab as kudu/127.11.114.195@KRBTEST.COM (short username kudu)
I20260807 06:42:15.700039 6827 webserver.cc:460] Webserver started at http://127.11.114.195:32967/ using document root <none> and password file <none>
I20260807 06:42:15.700338 6827 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:15.700418 6827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:15.700599 6827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:15.701885 6827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "f41d92682d214e06a64ec8a4c3a5d878"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "1ce5fc7a2c97eed9cb7cc200d477d9d9"
server_key_iv: "79c768f4d8cadb2b29e210c0036012c6"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.702301 6827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "f41d92682d214e06a64ec8a4c3a5d878"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "1ce5fc7a2c97eed9cb7cc200d477d9d9"
server_key_iv: "79c768f4d8cadb2b29e210c0036012c6"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.703873 6827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:42:15.704737 6851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.704953 6827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:15.705075 6827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "f41d92682d214e06a64ec8a4c3a5d878"
format_stamp: "Formatted at 2026-08-07 06:42:15 on dist-test-slave-gkw7"
server_key: "1ce5fc7a2c97eed9cb7cc200d477d9d9"
server_key_iv: "79c768f4d8cadb2b29e210c0036012c6"
server_key_version: "encryptionkey@0"
I20260807 06:42:15.705191 6827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:15.723421 6827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:15.725893 6827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:15.726074 6827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:15.726357 6827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:15.726729 6827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:15.726800 6827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.726835 6827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:15.726853 6827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:15.733052 6827 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:34181
I20260807 06:42:15.733134 6999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:34181 every 8 connection(s)
I20260807 06:42:15.733500 6827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:42:15.734356 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 6827
I20260807 06:42:15.734501 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:42:15.734753 11723 external_mini_cluster.cc:1428] Setting key 36cfd65006bdc4f3e156e82afe5df3f3
Aug 07 06:42:15 dist-test-slave-gkw7 krb5kdc[6275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084935, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.195@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:15.744222 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.734473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:54955 (local address 127.11.114.254:46371)
0807 06:42:15.734556 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:15.734558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:15.734833 (+ 275us) server_negotiation.cc:407] Connection header received
0807 06:42:15.735582 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:15.735583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:15.735595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:15.735639 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:15.736737 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.737263 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.738639 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.738958 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.739047 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.741392 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:15.741394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:15.741394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:15.741402 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:15.742742 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.743176 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.743178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.743179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.743254 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.743522 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.743523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.743523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.743655 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:15.743742 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:15.744036 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 06:42:15.744125 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:42:15.744444 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.734474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46371 (local address 127.11.114.195:54955)
0807 06:42:15.734666 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:15.734722 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:15.735449 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:15.735730 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:15.735735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:15.736418 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:15.736625 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.736630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.737371 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.737372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.738524 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.738527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.739054 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.739056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.739203 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.739739 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:15.739750 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:15.741285 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:15.742846 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:15.742853 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:15.742856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:15.743094 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:15.743326 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:15.743327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:15.743327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:15.743449 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:15.743731 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:15.743739 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:15.743891 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 06:42:15.744003 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:42:15.744174 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260807 06:42:15.745002 7000 heartbeater.cc:347] Connected to a master server at 127.11.114.254:46371
I20260807 06:42:15.745126 7000 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:15.745324 7000 heartbeater.cc:511] Master 127.11.114.254:46371 requested a full tablet report, sending...
I20260807 06:42:15.745848 6340 ts_manager.cc:194] Registered new tserver with Master: f41d92682d214e06a64ec8a4c3a5d878 (127.11.114.195:34181)
I20260807 06:42:15.746345 6340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.114.195@KRBTEST.COM'} at 127.11.114.195:54955
I20260807 06:42:15.746767 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:42:15.754891 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.748270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.11.114.254:46371)
0807 06:42:15.748332 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:15.748334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:15.748436 (+ 102us) server_negotiation.cc:407] Connection header received
0807 06:42:15.748482 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:15.748484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:15.748492 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:15.748547 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:15.749182 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.749759 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.750644 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.750832 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.750885 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.752235 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:15.752237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:15.752238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:15.752246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:15.753507 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.753965 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.753967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.753967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.754037 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:15.754271 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:15.754272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:15.754272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:15.754443 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:15.754572 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:15.754687 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 06:42:15.754777 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:42:15.756479 6340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:42:15.758214 6340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:42:15.764968 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.760552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:44787 (local address 127.0.0.1:37596)
0807 06:42:15.760979 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:15.760989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:15.761027 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:15.761607 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:15.761607 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:15.761609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:15.761652 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.761653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.762545 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.762546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.763745 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.763748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.764624 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.764625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.764714 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.764728 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 06:42:15.764826 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:42:15.764886 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":110,"threads_started":1}
I20260807 06:42:15.764986 7018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.760830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.11.114.194:44787)
0807 06:42:15.761359 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:15.761363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:15.761372 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:15.761409 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:15.761411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:15.761426 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:15.761500 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:15.761741 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.762445 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.763844 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.764513 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.764615 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.764683 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:15.764843 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 06:42:15.764890 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":93,"threads_started":1}
I20260807 06:42:15.765377 7015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.760553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34181 (local address 127.0.0.1:56436)
0807 06:42:15.761097 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:15.761106 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:15.761154 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:15.761425 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:15.761426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:15.761428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:15.761486 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.761489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.762935 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.762936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.763985 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.763987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.764977 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.764978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.765187 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.765192 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:15.765252 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:42:15.765299 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":41,"thread_start_us":102,"threads_started":1}
I20260807 06:42:15.765424 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.760641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56436 (local address 127.11.114.195:34181)
0807 06:42:15.761097 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:15.761101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:15.761147 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:42:15.761258 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:15.761261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:15.761279 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:15.761353 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:15.761590 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.762804 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.764077 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.764812 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.764887 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.764938 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:15.765283 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 06:42:15.765337 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":86,"threads_started":1}
I20260807 06:42:15.765851 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.760815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.11.114.193:39517)
0807 06:42:15.761032 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:15.761037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:15.761049 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:42:15.761091 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:15.761094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:15.761112 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:15.761198 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:15.761489 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.762831 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.764140 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.764825 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.764887 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.764938 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:15.765410 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 06:42:15.765644 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":112,"threads_started":1}
I20260807 06:42:15.765863 6723 tablet_service.cc:1467] Processing CreateTablet for tablet 0cfb19f496ee40468a15e16e914a41b9 (DEFAULT_TABLE table=test-table [id=8589c4d640174002b5a9c80de9f1c5cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:15.766008 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:15.760549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:39517 (local address 127.0.0.1:44752)
0807 06:42:15.760739 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:15.760791 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:15.760862 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:15.761303 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:15.761309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:15.761312 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:15.761387 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.761390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.762948 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.762949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.764058 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:15.764062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:15.764977 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:15.764978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:15.765043 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:15.765047 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:15.765316 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 06:42:15.765348 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":3,"thread_start_us":68,"threads_started":1}
I20260807 06:42:15.766294 6723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cfb19f496ee40468a15e16e914a41b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:15.766741 6534 tablet_service.cc:1467] Processing CreateTablet for tablet 0cfb19f496ee40468a15e16e914a41b9 (DEFAULT_TABLE table=test-table [id=8589c4d640174002b5a9c80de9f1c5cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:15.767098 6534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cfb19f496ee40468a15e16e914a41b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:15.769927 6915 tablet_service.cc:1467] Processing CreateTablet for tablet 0cfb19f496ee40468a15e16e914a41b9 (DEFAULT_TABLE table=test-table [id=8589c4d640174002b5a9c80de9f1c5cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:15.770144 7021 tablet_bootstrap.cc:492] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Bootstrap starting.
I20260807 06:42:15.770277 6915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cfb19f496ee40468a15e16e914a41b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:15.770682 7023 tablet_bootstrap.cc:492] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e: Bootstrap starting.
I20260807 06:42:15.770814 7021 tablet_bootstrap.cc:654] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:15.771353 7023 tablet_bootstrap.cc:654] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:15.771404 7021 log.cc:824] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:15.771924 7023 log.cc:824] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:15.772300 7021 tablet_bootstrap.cc:492] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: No bootstrap required, opened a new log
I20260807 06:42:15.772375 7021 ts_tablet_manager.cc:1397] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:42:15.772718 7023 tablet_bootstrap.cc:492] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e: No bootstrap required, opened a new log
I20260807 06:42:15.772789 7023 ts_tablet_manager.cc:1397] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:42:15.773705 7027 tablet_bootstrap.cc:492] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Bootstrap starting.
I20260807 06:42:15.774052 7021 raft_consensus.cc:348] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:15.774286 7021 raft_consensus.cc:374] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:15.774343 7021 raft_consensus.cc:729] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1fa53d913aa4084a3dcbb03a10da892, State: Initialized, Role: FOLLOWER
I20260807 06:42:15.774405 7027 tablet_bootstrap.cc:654] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:15.774528 7021 consensus_queue.cc:260] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:15.774881 6820 heartbeater.cc:503] Master 127.11.114.254:46371 was elected leader, sending a full tablet report...
I20260807 06:42:15.774771 7023 raft_consensus.cc:348] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:15.774969 7023 raft_consensus.cc:374] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:15.775013 7023 raft_consensus.cc:729] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 620ccf02ad7f4a79940f51dfaf18152e, State: Initialized, Role: FOLLOWER
I20260807 06:42:15.775064 7027 log.cc:824] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:15.775130 7023 consensus_queue.cc:260] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:15.775285 7021 ts_tablet_manager.cc:1428] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:15.775425 7023 ts_tablet_manager.cc:1428] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:15.775574 6625 heartbeater.cc:503] Master 127.11.114.254:46371 was elected leader, sending a full tablet report...
I20260807 06:42:15.775978 7027 tablet_bootstrap.cc:492] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: No bootstrap required, opened a new log
I20260807 06:42:15.776070 7027 ts_tablet_manager.cc:1397] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:42:15.777514 7027 raft_consensus.cc:348] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:15.777757 7027 raft_consensus.cc:374] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:15.777807 7027 raft_consensus.cc:729] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41d92682d214e06a64ec8a4c3a5d878, State: Initialized, Role: FOLLOWER
I20260807 06:42:15.777925 7027 consensus_queue.cc:260] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:15.778184 7027 ts_tablet_manager.cc:1428] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:42:15.778223 7000 heartbeater.cc:503] Master 127.11.114.254:46371 was elected leader, sending a full tablet report...
W20260807 06:42:15.817683 6821 tablet.cc:2369] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:42:15.915936 6627 tablet.cc:2369] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:42:15.984215 7001 tablet.cc:2369] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:42:16.007386 7028 raft_consensus.cc:482] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:42:16.007570 7028 raft_consensus.cc:504] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:16.007941 7028 leader_election.cc:288] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 620ccf02ad7f4a79940f51dfaf18152e (127.11.114.193:39517), f41d92682d214e06a64ec8a4c3a5d878 (127.11.114.195:34181)
I20260807 06:42:16.010947 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.008128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:39517 (local address 127.11.114.194:46923)
0807 06:42:16.008192 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.008205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.008255 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.008423 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.008424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.008425 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:16.008523 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.008525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.009256 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.009257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.010104 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.010106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.010658 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.010659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.010742 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.010746 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.010803 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.010850 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 06:42:16.010941 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.008177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:46923 (local address 127.11.114.193:39517)
0807 06:42:16.008228 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.008230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.008264 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:42:16.008325 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.008326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.008338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.008427 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:16.008644 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.009154 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.010191 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.010572 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.010636 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.010644 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.010818 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.010863 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:42:16.011034 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.008210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:34767 (local address 127.11.114.195:34181)
0807 06:42:16.008258 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.008260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.008450 (+ 190us) server_negotiation.cc:407] Connection header received
0807 06:42:16.008481 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.008482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.008491 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.008539 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:16.008676 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.009218 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.010273 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.010668 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.010750 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.010759 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.010921 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.010962 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:42:16.011117 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.008142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34181 (local address 127.11.114.194:34767)
0807 06:42:16.008334 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.008346 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.008425 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.008553 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.008554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.008556 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:16.008623 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.008624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.009351 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.009352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.010174 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.010177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.010750 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.010751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.010837 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.010841 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.010896 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.010932 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":117,"threads_started":1}
I20260807 06:42:16.011425 6560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cfb19f496ee40468a15e16e914a41b9" candidate_uuid: "f1fa53d913aa4084a3dcbb03a10da892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "620ccf02ad7f4a79940f51dfaf18152e" is_pre_election: true
I20260807 06:42:16.011466 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cfb19f496ee40468a15e16e914a41b9" candidate_uuid: "f1fa53d913aa4084a3dcbb03a10da892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41d92682d214e06a64ec8a4c3a5d878" is_pre_election: true
I20260807 06:42:16.011622 6560 raft_consensus.cc:2454] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1fa53d913aa4084a3dcbb03a10da892 in term 0.
I20260807 06:42:16.011641 6942 raft_consensus.cc:2454] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1fa53d913aa4084a3dcbb03a10da892 in term 0.
I20260807 06:42:16.011881 6659 leader_election.cc:302] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1fa53d913aa4084a3dcbb03a10da892, f41d92682d214e06a64ec8a4c3a5d878; no voters:
I20260807 06:42:16.012036 7028 raft_consensus.cc:2781] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:42:16.012094 7028 raft_consensus.cc:482] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:42:16.012156 7028 raft_consensus.cc:3037] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:16.012882 7028 raft_consensus.cc:504] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:16.013032 7028 leader_election.cc:288] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [CANDIDATE]: Term 1 election: Requested vote from peers 620ccf02ad7f4a79940f51dfaf18152e (127.11.114.193:39517), f41d92682d214e06a64ec8a4c3a5d878 (127.11.114.195:34181)
I20260807 06:42:16.013217 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cfb19f496ee40468a15e16e914a41b9" candidate_uuid: "f1fa53d913aa4084a3dcbb03a10da892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41d92682d214e06a64ec8a4c3a5d878"
I20260807 06:42:16.013223 6560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cfb19f496ee40468a15e16e914a41b9" candidate_uuid: "f1fa53d913aa4084a3dcbb03a10da892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "620ccf02ad7f4a79940f51dfaf18152e"
I20260807 06:42:16.013299 6942 raft_consensus.cc:3037] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:16.013321 6560 raft_consensus.cc:3037] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:16.014093 6942 raft_consensus.cc:2454] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1fa53d913aa4084a3dcbb03a10da892 in term 1.
I20260807 06:42:16.014154 6560 raft_consensus.cc:2454] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1fa53d913aa4084a3dcbb03a10da892 in term 1.
I20260807 06:42:16.014295 6659 leader_election.cc:302] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1fa53d913aa4084a3dcbb03a10da892, f41d92682d214e06a64ec8a4c3a5d878; no voters:
I20260807 06:42:16.014410 7028 raft_consensus.cc:2781] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:16.014559 7028 raft_consensus.cc:686] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 1 LEADER]: Becoming Leader. State: Replica: f1fa53d913aa4084a3dcbb03a10da892, State: Running, Role: LEADER
I20260807 06:42:16.014667 7028 consensus_queue.cc:237] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } }
I20260807 06:42:16.015655 6340 catalog_manager.cc:5495] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1fa53d913aa4084a3dcbb03a10da892 (127.11.114.194). New cstate: current_term: 1 leader_uuid: "f1fa53d913aa4084a3dcbb03a10da892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1fa53d913aa4084a3dcbb03a10da892" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44787 } health_report { overall_health: HEALTHY } } }
I20260807 06:42:16.048959 7018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.046509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37612 (local address 127.11.114.194:44787)
0807 06:42:16.046577 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.046579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.046589 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:42:16.046631 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.046631 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.046653 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.046717 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:16.046838 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.047350 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.048194 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.048395 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.048452 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.048608 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:16.048734 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:16.048843 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.048866 (+ 23us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.048885 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:42:16.050789 6560 raft_consensus.cc:1261] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 1 FOLLOWER]: Refusing update from remote peer f1fa53d913aa4084a3dcbb03a10da892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:16.050796 6942 raft_consensus.cc:1261] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 1 FOLLOWER]: Refusing update from remote peer f1fa53d913aa4084a3dcbb03a10da892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:16.051169 7034 consensus_queue.cc:1035] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "620ccf02ad7f4a79940f51dfaf18152e" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 39517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:16.051311 7028 consensus_queue.cc:1035] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41d92682d214e06a64ec8a4c3a5d878" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:16.053891 7039 mvcc.cc:205] Tried to move back new op lower bound from 7315803898061148160 to 7315803897916686336. Current Snapshot: MvccSnapshot[applied={T|T < 7315803898061148160}]
I20260807 06:42:16.056389 6340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:16.056497 6340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:16.057241 6340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae6333d2bf9548be959a4087cab0e541: Sending DeleteTablet for 3 replicas of tablet 0cfb19f496ee40468a15e16e914a41b9
I20260807 06:42:16.057611 6534 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cfb19f496ee40468a15e16e914a41b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:16 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:44752
I20260807 06:42:16.057703 6723 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cfb19f496ee40468a15e16e914a41b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:16 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:37596
I20260807 06:42:16.057693 6915 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cfb19f496ee40468a15e16e914a41b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:16 UTC) from {username='kudu', principal='kudu/127.11.114.254@KRBTEST.COM'} at 127.0.0.1:56436
I20260807 06:42:16.058025 7049 tablet_replica.cc:330] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e: stopping tablet replica
I20260807 06:42:16.058152 7049 raft_consensus.cc:2229] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:16.058213 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 6434
I20260807 06:42:16.058283 7049 raft_consensus.cc:2258] T 0cfb19f496ee40468a15e16e914a41b9 P 620ccf02ad7f4a79940f51dfaf18152e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:16.058445 7052 tablet_replica.cc:330] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: stopping tablet replica
I20260807 06:42:16.058525 7052 raft_consensus.cc:2229] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:42:16.058631 7052 raft_consensus.cc:2258] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:16.067149 7053 tablet_replica.cc:330] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: stopping tablet replica
I20260807 06:42:16.067238 7052 ts_tablet_manager.cc:1905] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:16.067309 7053 raft_consensus.cc:2229] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:16.067438 7053 raft_consensus.cc:2258] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:16.067904 7053 ts_tablet_manager.cc:1905] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:16.069046 7052 ts_tablet_manager.cc:1918] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:16.069130 7052 log.cc:1196] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/0cfb19f496ee40468a15e16e914a41b9
I20260807 06:42:16.069403 7052 ts_tablet_manager.cc:1939] T 0cfb19f496ee40468a15e16e914a41b9 P f1fa53d913aa4084a3dcbb03a10da892: Deleting consensus metadata
I20260807 06:42:16.069898 6321 catalog_manager.cc:4841] TS f1fa53d913aa4084a3dcbb03a10da892 (127.11.114.194:44787): tablet 0cfb19f496ee40468a15e16e914a41b9 (table test-table [id=8589c4d640174002b5a9c80de9f1c5cb]) successfully deleted
I20260807 06:42:16.069924 7053 ts_tablet_manager.cc:1918] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:16.070000 7053 log.cc:1196] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/0cfb19f496ee40468a15e16e914a41b9
I20260807 06:42:16.070313 7053 ts_tablet_manager.cc:1939] T 0cfb19f496ee40468a15e16e914a41b9 P f41d92682d214e06a64ec8a4c3a5d878: Deleting consensus metadata
I20260807 06:42:16.071045 6318 catalog_manager.cc:4841] TS f41d92682d214e06a64ec8a4c3a5d878 (127.11.114.195:34181): tablet 0cfb19f496ee40468a15e16e914a41b9 (table test-table [id=8589c4d640174002b5a9c80de9f1c5cb]) successfully deleted
W20260807 06:42:16.073200 6322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:39517 (error 108)
I20260807 06:42:16.073614 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 6637
I20260807 06:42:16.078320 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.078151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:39517 (local address 127.0.0.1:44768)
0807 06:42:16.078225 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.078280 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:39517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 06:42:16.078445 6322 catalog_manager.cc:4584] TS 620ccf02ad7f4a79940f51dfaf18152e (127.11.114.193:39517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0cfb19f496ee40468a15e16e914a41b9: Network error: Client connection negotiation failed: client connection to 127.11.114.193:39517: connect: Connection refused (error 111)
I20260807 06:42:16.083315 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 6827
I20260807 06:42:16.092883 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 6294
2026-08-07T06:42:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2324 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T06:42:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:16Z Disabled control of system clock
I20260807 06:42:16.126611 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:43439
--webserver_interface=127.11.114.254
--webserver_port=0
--builtin_ntp_servers=127.11.114.212:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:43439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:16.207959 7060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:16.208180 7060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:16.208201 7060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:16.209277 7060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:16.209311 7060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:16.209323 7060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:16.209335 7060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:16.209347 7060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:16.210664 7060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:43439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:43439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:16.210873 7060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:16.211125 7060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:16.216440 7066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:16.216508 7068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:16.216542 7060 server_base.cc:1034] running on GCE node
W20260807 06:42:16.216440 7070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:16.217018 7060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:16.217348 7060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:16.218524 7060 hybrid_clock.cc:648] HybridClock initialized: now 1786084936218483 us; error 56 us; skew 500 ppm
I20260807 06:42:16.219914 7060 webserver.cc:460] Webserver started at http://127.11.114.254:34477/ using document root <none> and password file <none>
I20260807 06:42:16.220223 7060 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:16.220279 7060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:16.220468 7060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:16.221750 7060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "2bf9c14afcb94b78850c0f66e46f069d"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "30cae660e9afdcd20219aebcbad7d900"
server_key_iv: "5b935ff291bc9ef5c4b2de35277920b5"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.222172 7060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "2bf9c14afcb94b78850c0f66e46f069d"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "30cae660e9afdcd20219aebcbad7d900"
server_key_iv: "5b935ff291bc9ef5c4b2de35277920b5"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.223594 7060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:42:16.224359 7079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.224548 7060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:42:16.224649 7060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "2bf9c14afcb94b78850c0f66e46f069d"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "30cae660e9afdcd20219aebcbad7d900"
server_key_iv: "5b935ff291bc9ef5c4b2de35277920b5"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.224725 7060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:16.237051 7060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:16.237403 7060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:16.237596 7060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:16.241703 7060 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:43439
I20260807 06:42:16.241760 7155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:43439 every 8 connection(s)
I20260807 06:42:16.242169 7060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:16.243176 7156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:16.245496 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d: Bootstrap starting.
I20260807 06:42:16.246155 7156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:16.246469 7156 log.cc:824] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:16.247267 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d: No bootstrap required, opened a new log
I20260807 06:42:16.249110 7156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf9c14afcb94b78850c0f66e46f069d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43439 } }
I20260807 06:42:16.249257 7156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:16.249284 7156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf9c14afcb94b78850c0f66e46f069d, State: Initialized, Role: FOLLOWER
I20260807 06:42:16.249475 7156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf9c14afcb94b78850c0f66e46f069d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43439 } }
I20260807 06:42:16.249604 7156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:16.249655 7156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:16.249712 7156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:16.250291 7156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf9c14afcb94b78850c0f66e46f069d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43439 } }
I20260807 06:42:16.250408 7156 leader_election.cc:302] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bf9c14afcb94b78850c0f66e46f069d; no voters:
I20260807 06:42:16.250614 7156 leader_election.cc:288] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:16.250679 7160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:16.250937 7156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:16.250918 7160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [term 1 LEADER]: Becoming Leader. State: Replica: 2bf9c14afcb94b78850c0f66e46f069d, State: Running, Role: LEADER
I20260807 06:42:16.251099 7160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf9c14afcb94b78850c0f66e46f069d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43439 } }
I20260807 06:42:16.251381 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 7060
I20260807 06:42:16.251493 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:16.251695 7161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bf9c14afcb94b78850c0f66e46f069d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf9c14afcb94b78850c0f66e46f069d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43439 } } }
I20260807 06:42:16.251754 11723 external_mini_cluster.cc:1428] Setting key 1ae0cc4ac385f6f82833849690fdf32a
I20260807 06:42:16.251773 7161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:16.252046 7172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:16.252198 7162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bf9c14afcb94b78850c0f66e46f069d. Latest consensus state: current_term: 1 leader_uuid: "2bf9c14afcb94b78850c0f66e46f069d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf9c14afcb94b78850c0f66e46f069d" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 43439 } } }
I20260807 06:42:16.252355 7162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:16.252645 7172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:16.254585 7172 catalog_manager.cc:1347] Generated new cluster ID: 42240f16103e4eed9033fc44e0e93312
I20260807 06:42:16.254645 7172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:16.259434 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.252516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.11.114.254:43439)
0807 06:42:16.253509 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.253516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.253590 (+ 74us) server_negotiation.cc:407] Connection header received
0807 06:42:16.253698 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.253703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.253756 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.254095 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:16.254368 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.255766 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.256939 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.257602 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.257739 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.258712 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:16.258716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:16.258716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:16.258931 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:16.258975 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:16.258991 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:16.259066 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.259141 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.259221 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":84,"threads_started":1}
I20260807 06:42:16.261039 7172 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:16.261588 7172 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:16.265112 7172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d: Generated new TSK 0
I20260807 06:42:16.265302 7172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:16.268445 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.193:0
--local_ip_for_outbound_sockets=127.11.114.193
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43439
--builtin_ntp_servers=127.11.114.212:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:16.352826 7192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:16.353034 7192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:16.353055 7192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:16.354336 7192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:16.354378 7192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:16.354404 7192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.193
I20260807 06:42:16.355747 7192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.114.193
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.193
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:16.355994 7192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:16.356344 7192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:16.362201 7197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:16.362304 7198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:16.362399 7201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:16.362702 7192 server_base.cc:1034] running on GCE node
I20260807 06:42:16.362947 7192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:16.363195 7192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:16.364398 7192 hybrid_clock.cc:648] HybridClock initialized: now 1786084936364369 us; error 43 us; skew 500 ppm
I20260807 06:42:16.366082 7192 webserver.cc:460] Webserver started at http://127.11.114.193:38099/ using document root <none> and password file <none>
I20260807 06:42:16.366353 7192 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:16.366427 7192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:16.366604 7192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:16.367722 7192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/data/instance:
uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "61172e20d367ce16accb0b78868a7cad"
server_key_iv: "7c72e88426ca518db89704034024f560"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.368115 7192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance:
uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "61172e20d367ce16accb0b78868a7cad"
server_key_iv: "7c72e88426ca518db89704034024f560"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.369658 7192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:16.370532 7211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.370733 7192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:16.370846 7192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/wal
uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "61172e20d367ce16accb0b78868a7cad"
server_key_iv: "7c72e88426ca518db89704034024f560"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.370929 7192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:16.381749 7192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:16.382063 7192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:16.382227 7192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:16.382462 7192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:16.382817 7192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:16.382885 7192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.382931 7192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:16.382962 7192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.389359 7192 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.193:33143
I20260807 06:42:16.389487 7372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.193:33143 every 8 connection(s)
I20260807 06:42:16.389868 7192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/data/info.pb
I20260807 06:42:16.394032 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 7192
I20260807 06:42:16.394207 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-0/wal/instance
I20260807 06:42:16.394424 11723 external_mini_cluster.cc:1428] Setting key 4b3d040af94de43c86e12152aca05687
I20260807 06:42:16.395552 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.194:0
--local_ip_for_outbound_sockets=127.11.114.194
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43439
--builtin_ntp_servers=127.11.114.212:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:42:16.395634 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.390655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.193:41855 (local address 127.11.114.254:43439)
0807 06:42:16.390755 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.390757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.390933 (+ 176us) server_negotiation.cc:407] Connection header received
0807 06:42:16.391433 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.391434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.391445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.391497 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:16.392156 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.392647 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.393898 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.394086 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.394142 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.395301 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:16.395302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:16.395303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:16.395308 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:16.395331 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:16.395335 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:16.395402 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.395495 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.395533 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 06:42:16.395897 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.390594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43439 (local address 127.11.114.193:41855)
0807 06:42:16.390808 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.390842 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.391342 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.391567 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.391571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.391579 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:16.392081 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.392085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.392754 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.392755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.393787 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.393789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.394224 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.394225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.394406 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.395121 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:16.395122 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:16.395160 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:16.395193 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:16.395404 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:16.395411 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:16.395415 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.395495 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.395602 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260807 06:42:16.396425 7373 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43439
I20260807 06:42:16.396519 7373 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:16.396735 7373 heartbeater.cc:511] Master 127.11.114.254:43439 requested a full tablet report, sending...
I20260807 06:42:16.397271 7111 ts_manager.cc:194] Registered new tserver with Master: 320ead95a9bb4e52b2f7bb088ecdbb3d (127.11.114.193:33143)
I20260807 06:42:16.398039 7111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.193:41855
W20260807 06:42:16.480944 7377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:16.481164 7377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:16.481201 7377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:16.482403 7377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:16.482465 7377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:16.482509 7377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.194
I20260807 06:42:16.483856 7377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.114.194
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.194
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:16.484105 7377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:16.484404 7377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:16.491068 7388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:16.491163 7394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:16.491079 7390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:16.491520 7377 server_base.cc:1034] running on GCE node
I20260807 06:42:16.491732 7377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:16.491992 7377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:16.493145 7377 hybrid_clock.cc:648] HybridClock initialized: now 1786084936493128 us; error 34 us; skew 500 ppm
I20260807 06:42:16.494494 7377 webserver.cc:460] Webserver started at http://127.11.114.194:36419/ using document root <none> and password file <none>
I20260807 06:42:16.494751 7377 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:16.494798 7377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:16.494935 7377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:16.496001 7377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/data/instance:
uuid: "5255f69d7d8f4be2becdd26696e13757"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "f7cb89cca783a9b117e7960b2479db3b"
server_key_iv: "9c2775f841daace1c1b34c2fba0150ac"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.496364 7377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance:
uuid: "5255f69d7d8f4be2becdd26696e13757"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "f7cb89cca783a9b117e7960b2479db3b"
server_key_iv: "9c2775f841daace1c1b34c2fba0150ac"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.497779 7377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:16.498615 7402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.498857 7377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:16.498960 7377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/wal
uuid: "5255f69d7d8f4be2becdd26696e13757"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "f7cb89cca783a9b117e7960b2479db3b"
server_key_iv: "9c2775f841daace1c1b34c2fba0150ac"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.499049 7377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:16.507366 7377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:16.507668 7377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:16.507818 7377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:16.508045 7377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:16.508430 7377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:16.508481 7377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.508514 7377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:16.508535 7377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.514725 7377 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.194:44333
I20260807 06:42:16.514820 7554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.194:44333 every 8 connection(s)
I20260807 06:42:16.515122 7377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/data/info.pb
I20260807 06:42:16.520859 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.515935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.194:40065 (local address 127.11.114.254:43439)
0807 06:42:16.516006 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.516008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.516235 (+ 227us) server_negotiation.cc:407] Connection header received
0807 06:42:16.516702 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.516703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.516718 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.516805 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:16.517461 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.518354 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.519286 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.519648 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.519733 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.520532 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:16.520533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:16.520534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:16.520540 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:16.520557 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:16.520562 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:16.520627 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.520708 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.520738 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:42:16.520910 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.515897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43439 (local address 127.11.114.194:40065)
0807 06:42:16.516100 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.516140 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.516610 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.516883 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.516886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.516894 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:16.517376 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.517380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.518449 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.518450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.519199 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.519201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.519735 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.519736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.519838 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.520393 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:16.520393 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:16.520420 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:16.520442 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:16.520631 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:16.520635 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:16.520637 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.520687 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.520762 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260807 06:42:16.521263 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 7377
I20260807 06:42:16.521402 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/wal/instance
I20260807 06:42:16.521596 7555 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43439
I20260807 06:42:16.521649 11723 external_mini_cluster.cc:1428] Setting key dde1a3e68da9839b3dcdbc210e53f111
I20260807 06:42:16.521709 7555 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:16.521961 7555 heartbeater.cc:511] Master 127.11.114.254:43439 requested a full tablet report, sending...
I20260807 06:42:16.522374 7111 ts_manager.cc:194] Registered new tserver with Master: 5255f69d7d8f4be2becdd26696e13757 (127.11.114.194:44333)
I20260807 06:42:16.522730 7111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.194:40065
I20260807 06:42:16.523123 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.114.195:0
--local_ip_for_outbound_sockets=127.11.114.195
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43439
--builtin_ntp_servers=127.11.114.212:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:42:16.607512 7562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:16.607726 7562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:16.607748 7562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:16.608893 7562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:16.608934 7562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:16.608959 7562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.114.195
I20260807 06:42:16.610303 7562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.114.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.114.195
--webserver_port=0
--tserver_master_addrs=127.11.114.254:43439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.114.195
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:16.610543 7562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:16.610850 7562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:16.615969 7568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:16.615975 7569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:16.615979 7572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:16.616182 7562 server_base.cc:1034] running on GCE node
I20260807 06:42:16.616472 7562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:16.616729 7562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:16.617885 7562 hybrid_clock.cc:648] HybridClock initialized: now 1786084936617869 us; error 47 us; skew 500 ppm
I20260807 06:42:16.619177 7562 webserver.cc:460] Webserver started at http://127.11.114.195:32917/ using document root <none> and password file <none>
I20260807 06:42:16.619444 7562 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:16.619513 7562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:16.619665 7562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:16.620792 7562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/data/instance:
uuid: "9430fdbdc164417a86f6299644cc55a3"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "b6e075d52809e46fcdd6e1f36e381cf1"
server_key_iv: "f3184a02f923176d29b83ff7ba71e513"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.621165 7562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance:
uuid: "9430fdbdc164417a86f6299644cc55a3"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "b6e075d52809e46fcdd6e1f36e381cf1"
server_key_iv: "f3184a02f923176d29b83ff7ba71e513"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.622560 7562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:16.623327 7583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.623520 7562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:16.623615 7562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/wal
uuid: "9430fdbdc164417a86f6299644cc55a3"
format_stamp: "Formatted at 2026-08-07 06:42:16 on dist-test-slave-gkw7"
server_key: "b6e075d52809e46fcdd6e1f36e381cf1"
server_key_iv: "f3184a02f923176d29b83ff7ba71e513"
server_key_version: "encryptionkey@0"
I20260807 06:42:16.623705 7562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:16.630674 7562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:16.630962 7562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:16.631114 7562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:16.631350 7562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:42:16.631675 7562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:42:16.631735 7562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.631770 7562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:42:16.631798 7562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:16.638010 7562 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.195:34693
I20260807 06:42:16.638204 7740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.195:34693 every 8 connection(s)
I20260807 06:42:16.638458 7562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/data/info.pb
I20260807 06:42:16.639348 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 7562
I20260807 06:42:16.639459 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/wal/instance
I20260807 06:42:16.639659 11723 external_mini_cluster.cc:1428] Setting key 9cca5fff0223ce45e7fccbd9441236db
I20260807 06:42:16.643976 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.639233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:59195 (local address 127.11.114.254:43439)
0807 06:42:16.639286 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.639288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.639542 (+ 254us) server_negotiation.cc:407] Connection header received
0807 06:42:16.640129 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.640130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.640143 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.640202 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:16.640955 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.641472 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.642500 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.642719 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.642813 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.643669 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:16.643670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:16.643671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:16.643676 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:16.643691 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:16.643695 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:16.643760 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.643867 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.643897 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:42:16.644181 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.639232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:43439 (local address 127.11.114.195:59195)
0807 06:42:16.639423 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.639457 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.640030 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.640284 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.640287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.640295 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:16.640861 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.640865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.641588 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.641589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.642405 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.642408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.642824 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.642825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.642955 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.643526 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:42:16.643527 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:16.643555 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:42:16.643577 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:16.643764 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:16.643769 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:16.643772 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.643853 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.643955 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
I20260807 06:42:16.644601 7741 heartbeater.cc:347] Connected to a master server at 127.11.114.254:43439
I20260807 06:42:16.644677 7741 heartbeater.cc:464] Registering TS with master...
I20260807 06:42:16.644876 7741 heartbeater.cc:511] Master 127.11.114.254:43439 requested a full tablet report, sending...
I20260807 06:42:16.645287 7111 ts_manager.cc:194] Registered new tserver with Master: 9430fdbdc164417a86f6299644cc55a3 (127.11.114.195:34693)
I20260807 06:42:16.645769 7111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.114.195:59195
I20260807 06:42:16.651288 11723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:42:16.655865 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.652853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.11.114.254:43439)
0807 06:42:16.652912 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.652914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.652994 (+ 80us) server_negotiation.cc:407] Connection header received
0807 06:42:16.653063 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.653064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.653076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.653148 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:16.653345 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.653868 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.654653 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.654843 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.654915 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.655603 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:16.655605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:16.655605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:16.655610 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:16.655623 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:16.655627 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:16.655695 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.655753 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.655779 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 06:42:16.657297 7111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:42:16.658751 7111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:42:16.664887 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.660759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:33143 (local address 127.0.0.1:36430)
0807 06:42:16.660928 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.660970 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.661037 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.661315 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.661318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.661320 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:16.661385 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.661386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.662522 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.662523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.663835 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.663837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.664594 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.664595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.664667 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.664680 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.664764 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.664815 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260807 06:42:16.664902 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.660953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.11.114.193:33143)
0807 06:42:16.661143 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.661148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.661160 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:42:16.661204 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.661207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.661224 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.661299 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:16.661486 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.662437 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.663917 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.664495 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.664594 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.664686 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.664777 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.664815 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260807 06:42:16.665194 7762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.661205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.11.114.194:44333)
0807 06:42:16.661671 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.661675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.661684 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:16.661720 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.661722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.661738 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.661832 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:16.661935 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.662980 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.663902 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.664554 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.664624 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.664702 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.665066 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.665111 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":91,"threads_started":1}
I20260807 06:42:16.665232 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.660781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.195:34693 (local address 127.0.0.1:38198)
0807 06:42:16.661165 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.661177 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.661246 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.661679 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.661680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.661682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:16.661722 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.661724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.663039 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.663040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.664076 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.664079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.665011 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.665012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.665068 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.665072 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.665129 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.665162 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":92,"threads_started":1}
I20260807 06:42:16.665000 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.660774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:44333 (local address 127.0.0.1:55692)
0807 06:42:16.661146 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.661155 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.661195 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.661829 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.661830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.661832 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:16.661873 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.661874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.663068 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.663069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.663801 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.663803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.664752 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.664753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.664818 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.664823 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.664881 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.664924 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":94,"threads_started":1}
I20260807 06:42:16.665578 7761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.661025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38198 (local address 127.11.114.195:34693)
0807 06:42:16.661410 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.661414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.661425 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:42:16.661465 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.661467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.661482 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.661621 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:16.661836 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.662947 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.664159 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.664848 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.664908 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.664978 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.665144 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.665192 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":83,"threads_started":1}
I20260807 06:42:16.665966 7281 tablet_service.cc:1467] Processing CreateTablet for tablet abfe9e72670f4f8192a9b6c77dde9da7 (DEFAULT_TABLE table=test-table [id=86691b8bf9d54b3285c546ad6830b641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:16.666049 7463 tablet_service.cc:1467] Processing CreateTablet for tablet abfe9e72670f4f8192a9b6c77dde9da7 (DEFAULT_TABLE table=test-table [id=86691b8bf9d54b3285c546ad6830b641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:16.666181 7643 tablet_service.cc:1467] Processing CreateTablet for tablet abfe9e72670f4f8192a9b6c77dde9da7 (DEFAULT_TABLE table=test-table [id=86691b8bf9d54b3285c546ad6830b641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:42:16.666376 7281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abfe9e72670f4f8192a9b6c77dde9da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:16.666431 7463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abfe9e72670f4f8192a9b6c77dde9da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:16.666468 7643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abfe9e72670f4f8192a9b6c77dde9da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:16.668887 7764 tablet_bootstrap.cc:492] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d: Bootstrap starting.
I20260807 06:42:16.669457 7764 tablet_bootstrap.cc:654] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:16.669835 7764 log.cc:824] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:16.670112 7766 tablet_bootstrap.cc:492] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Bootstrap starting.
I20260807 06:42:16.670648 7766 tablet_bootstrap.cc:654] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:16.670689 7764 tablet_bootstrap.cc:492] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d: No bootstrap required, opened a new log
I20260807 06:42:16.670743 7764 ts_tablet_manager.cc:1397] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:42:16.670953 7766 log.cc:824] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:16.671303 7765 tablet_bootstrap.cc:492] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: Bootstrap starting.
I20260807 06:42:16.671626 7766 tablet_bootstrap.cc:492] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: No bootstrap required, opened a new log
I20260807 06:42:16.671701 7766 ts_tablet_manager.cc:1397] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:42:16.672798 7764 raft_consensus.cc:348] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.672994 7764 raft_consensus.cc:374] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:16.673043 7764 raft_consensus.cc:729] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320ead95a9bb4e52b2f7bb088ecdbb3d, State: Initialized, Role: FOLLOWER
I20260807 06:42:16.673154 7764 consensus_queue.cc:260] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.673472 7764 ts_tablet_manager.cc:1428] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:16.673516 7373 heartbeater.cc:503] Master 127.11.114.254:43439 was elected leader, sending a full tablet report...
I20260807 06:42:16.674036 7766 raft_consensus.cc:348] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.674221 7766 raft_consensus.cc:374] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:16.674249 7765 tablet_bootstrap.cc:654] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:16.674268 7766 raft_consensus.cc:729] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5255f69d7d8f4be2becdd26696e13757, State: Initialized, Role: FOLLOWER
I20260807 06:42:16.674401 7766 consensus_queue.cc:260] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.674556 7765 log.cc:824] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:16.674746 7555 heartbeater.cc:503] Master 127.11.114.254:43439 was elected leader, sending a full tablet report...
I20260807 06:42:16.674681 7766 ts_tablet_manager.cc:1428] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:42:16.675354 7765 tablet_bootstrap.cc:492] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: No bootstrap required, opened a new log
I20260807 06:42:16.675400 7765 ts_tablet_manager.cc:1397] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:42:16.677404 7765 raft_consensus.cc:348] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.677520 7765 raft_consensus.cc:374] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:16.677603 7765 raft_consensus.cc:729] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9430fdbdc164417a86f6299644cc55a3, State: Initialized, Role: FOLLOWER
I20260807 06:42:16.677711 7765 consensus_queue.cc:260] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.677925 7765 ts_tablet_manager.cc:1428] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 06:42:16.678010 7741 heartbeater.cc:503] Master 127.11.114.254:43439 was elected leader, sending a full tablet report...
I20260807 06:42:16.741497 7773 raft_consensus.cc:482] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:42:16.741648 7773 raft_consensus.cc:504] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.742056 7773 leader_election.cc:288] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 320ead95a9bb4e52b2f7bb088ecdbb3d (127.11.114.193:33143), 5255f69d7d8f4be2becdd26696e13757 (127.11.114.194:44333)
I20260807 06:42:16.745047 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.742258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.194:44333 (local address 127.11.114.195:50637)
0807 06:42:16.742343 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.742357 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.742417 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.742604 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.742605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.742608 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:16.742692 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.742694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.743409 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.743410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.744216 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.744218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.744790 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.744791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.744871 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.744875 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.744934 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.744970 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 06:42:16.745072 7762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.742348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:50637 (local address 127.11.114.194:44333)
0807 06:42:16.742393 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.742395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.742404 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:16.742491 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.742492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.742503 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.742560 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:16.742779 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.743324 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.744319 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.744710 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.744769 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.744776 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.744948 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.744988 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:42:16.745193 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.742376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.114.195:53641 (local address 127.11.114.193:33143)
0807 06:42:16.742429 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.742431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.742610 (+ 179us) server_negotiation.cc:407] Connection header received
0807 06:42:16.742648 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.742649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.742662 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.742736 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:42:16.742926 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.743471 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.744399 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.744799 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.744871 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.744878 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.745063 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.745117 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:42:16.745350 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.742258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:33143 (local address 127.11.114.195:53641)
0807 06:42:16.742540 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.742549 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:16.742600 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:16.742778 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:16.742779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:16.742781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:42:16.742849 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.742851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.743555 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.743556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.744314 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:16.744316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.744886 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.744887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:16.744980 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.744983 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:42:16.745042 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:42:16.745076 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":41,"thread_start_us":150,"threads_started":1}
I20260807 06:42:16.745461 7492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abfe9e72670f4f8192a9b6c77dde9da7" candidate_uuid: "9430fdbdc164417a86f6299644cc55a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5255f69d7d8f4be2becdd26696e13757" is_pre_election: true
I20260807 06:42:16.745683 7492 raft_consensus.cc:2454] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9430fdbdc164417a86f6299644cc55a3 in term 0.
I20260807 06:42:16.745761 7313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abfe9e72670f4f8192a9b6c77dde9da7" candidate_uuid: "9430fdbdc164417a86f6299644cc55a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" is_pre_election: true
I20260807 06:42:16.745916 7313 raft_consensus.cc:2454] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9430fdbdc164417a86f6299644cc55a3 in term 0.
I20260807 06:42:16.745956 7590 leader_election.cc:302] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5255f69d7d8f4be2becdd26696e13757, 9430fdbdc164417a86f6299644cc55a3; no voters:
I20260807 06:42:16.746124 7773 raft_consensus.cc:2781] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:42:16.746176 7773 raft_consensus.cc:482] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:42:16.746220 7773 raft_consensus.cc:3037] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:16.746953 7773 raft_consensus.cc:504] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.747109 7773 leader_election.cc:288] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [CANDIDATE]: Term 1 election: Requested vote from peers 320ead95a9bb4e52b2f7bb088ecdbb3d (127.11.114.193:33143), 5255f69d7d8f4be2becdd26696e13757 (127.11.114.194:44333)
I20260807 06:42:16.747232 7313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abfe9e72670f4f8192a9b6c77dde9da7" candidate_uuid: "9430fdbdc164417a86f6299644cc55a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d"
I20260807 06:42:16.747305 7492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abfe9e72670f4f8192a9b6c77dde9da7" candidate_uuid: "9430fdbdc164417a86f6299644cc55a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5255f69d7d8f4be2becdd26696e13757"
I20260807 06:42:16.747342 7313 raft_consensus.cc:3037] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:16.747383 7492 raft_consensus.cc:3037] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:16.748097 7492 raft_consensus.cc:2454] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9430fdbdc164417a86f6299644cc55a3 in term 1.
I20260807 06:42:16.748102 7313 raft_consensus.cc:2454] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9430fdbdc164417a86f6299644cc55a3 in term 1.
I20260807 06:42:16.748306 7585 leader_election.cc:302] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320ead95a9bb4e52b2f7bb088ecdbb3d, 9430fdbdc164417a86f6299644cc55a3; no voters:
I20260807 06:42:16.748418 7773 raft_consensus.cc:2781] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:16.748584 7773 raft_consensus.cc:686] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 1 LEADER]: Becoming Leader. State: Replica: 9430fdbdc164417a86f6299644cc55a3, State: Running, Role: LEADER
I20260807 06:42:16.748683 7773 consensus_queue.cc:237] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } }
I20260807 06:42:16.749564 7111 catalog_manager.cc:5495] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9430fdbdc164417a86f6299644cc55a3 (127.11.114.195). New cstate: current_term: 1 leader_uuid: "9430fdbdc164417a86f6299644cc55a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9430fdbdc164417a86f6299644cc55a3" member_type: VOTER last_known_addr { host: "127.11.114.195" port: 34693 } health_report { overall_health: HEALTHY } } }
I20260807 06:42:16.761132 7761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.758926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38214 (local address 127.11.114.195:34693)
0807 06:42:16.758975 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:16.758977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:16.758984 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:42:16.759014 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:16.759015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:16.759034 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:16.759112 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:16.759233 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.759784 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.760487 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:16.760672 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:16.760725 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:16.760856 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:16.760950 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:16.761011 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:16.761040 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 06:42:16.761063 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:42:16.762902 7492 raft_consensus.cc:1261] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 1 FOLLOWER]: Refusing update from remote peer 9430fdbdc164417a86f6299644cc55a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:16.762908 7313 raft_consensus.cc:1261] T abfe9e72670f4f8192a9b6c77dde9da7 P 320ead95a9bb4e52b2f7bb088ecdbb3d [term 1 FOLLOWER]: Refusing update from remote peer 9430fdbdc164417a86f6299644cc55a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:16.763243 7778 consensus_queue.cc:1035] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5255f69d7d8f4be2becdd26696e13757" member_type: VOTER last_known_addr { host: "127.11.114.194" port: 44333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:16.763370 7773 consensus_queue.cc:1035] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "320ead95a9bb4e52b2f7bb088ecdbb3d" member_type: VOTER last_known_addr { host: "127.11.114.193" port: 33143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:42:16.765666 7557 tablet.cc:2369] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:42:16.766065 7782 mvcc.cc:205] Tried to move back new op lower bound from 7315803900978384896 to 7315803900923400192. Current Snapshot: MvccSnapshot[applied={T|T < 7315803900978384896}]
I20260807 06:42:16.766258 7781 mvcc.cc:205] Tried to move back new op lower bound from 7315803900978384896 to 7315803900923400192. Current Snapshot: MvccSnapshot[applied={T|T < 7315803900978384896}]
I20260807 06:42:16.766463 7783 mvcc.cc:205] Tried to move back new op lower bound from 7315803900978384896 to 7315803900923400192. Current Snapshot: MvccSnapshot[applied={T|T < 7315803900978384896}]
I20260807 06:42:16.768446 7110 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:16.768554 7110 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:42:16.769312 7110 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bf9c14afcb94b78850c0f66e46f069d: Sending DeleteTablet for 3 replicas of tablet abfe9e72670f4f8192a9b6c77dde9da7
I20260807 06:42:16.769670 7281 tablet_service.cc:1514] Processing DeleteTablet for tablet abfe9e72670f4f8192a9b6c77dde9da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:16 UTC) from {username='slave'} at 127.0.0.1:36430
I20260807 06:42:16.769680 7463 tablet_service.cc:1514] Processing DeleteTablet for tablet abfe9e72670f4f8192a9b6c77dde9da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:16 UTC) from {username='slave'} at 127.0.0.1:55692
I20260807 06:42:16.769671 7643 tablet_service.cc:1514] Processing DeleteTablet for tablet abfe9e72670f4f8192a9b6c77dde9da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:42:16 UTC) from {username='slave'} at 127.0.0.1:38198
I20260807 06:42:16.769968 7793 tablet_replica.cc:330] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: stopping tablet replica
I20260807 06:42:16.770064 7793 raft_consensus.cc:2229] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:42:16.770221 7793 raft_consensus.cc:2258] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:16.770248 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 7192
I20260807 06:42:16.772761 7793 ts_tablet_manager.cc:1905] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:16.774542 7793 ts_tablet_manager.cc:1918] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:16.774629 7793 log.cc:1196] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-2/wal/wals/abfe9e72670f4f8192a9b6c77dde9da7
I20260807 06:42:16.774891 7793 ts_tablet_manager.cc:1939] T abfe9e72670f4f8192a9b6c77dde9da7 P 9430fdbdc164417a86f6299644cc55a3: Deleting consensus metadata
I20260807 06:42:16.775316 7085 catalog_manager.cc:4841] TS 9430fdbdc164417a86f6299644cc55a3 (127.11.114.195:34693): tablet abfe9e72670f4f8192a9b6c77dde9da7 (table test-table [id=86691b8bf9d54b3285c546ad6830b641]) successfully deleted
I20260807 06:42:16.779332 7797 tablet_replica.cc:330] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: stopping tablet replica
I20260807 06:42:16.779464 7797 raft_consensus.cc:2229] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:42:16.779578 7797 raft_consensus.cc:2258] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:42:16.780102 7797 ts_tablet_manager.cc:1905] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:42:16.781704 7797 ts_tablet_manager.cc:1918] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:42:16.781796 7797 log.cc:1196] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Deleting WAL directory at /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084684348861-11723-0/minicluster-data/ts-1/wal/wals/abfe9e72670f4f8192a9b6c77dde9da7
I20260807 06:42:16.782044 7797 ts_tablet_manager.cc:1939] T abfe9e72670f4f8192a9b6c77dde9da7 P 5255f69d7d8f4be2becdd26696e13757: Deleting consensus metadata
I20260807 06:42:16.782454 7087 catalog_manager.cc:4841] TS 5255f69d7d8f4be2becdd26696e13757 (127.11.114.194:44333): tablet abfe9e72670f4f8192a9b6c77dde9da7 (table test-table [id=86691b8bf9d54b3285c546ad6830b641]) successfully deleted
W20260807 06:42:16.785293 7080 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.114.193:33143 (error 108)
I20260807 06:42:16.785836 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:16.785676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.193:33143 (local address 127.0.0.1:36444)
0807 06:42:16.785741 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:16.785799 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.193:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 06:42:16.785972 7080 catalog_manager.cc:4584] TS 320ead95a9bb4e52b2f7bb088ecdbb3d (127.11.114.193:33143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet abfe9e72670f4f8192a9b6c77dde9da7: Network error: Client connection negotiation failed: client connection to 127.11.114.193:33143: connect: Connection refused (error 111)
I20260807 06:42:16.786230 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 7377
I20260807 06:42:16.799507 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 7562
I20260807 06:42:16.809778 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 7060
2026-08-07T06:42:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (715 ms)
[----------] 6 tests from EncryptionPolicyTest (8964 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:16 dist-test-slave-gkw7 krb5kdc[7802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:16 dist-test-slave-gkw7 krb5kdc[7802](info): set up 2 sockets
Aug 07 06:42:16 dist-test-slave-gkw7 krb5kdc[7802](info): commencing operation
krb5kdc: starting...
W20260807 06:42:18.026321 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:18 dist-test-slave-gkw7 krb5kdc[7802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:18.154678 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:18.236549 7818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:18.236792 7818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:18.236850 7818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:18.238075 7818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:18.238143 7818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:18.238168 7818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:18.238190 7818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:18.238219 7818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:18.239694 7818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40207
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:18.240000 7818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:18.240231 7818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:18.245066 7827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:18.245057 7832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:18.245077 7829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:18.245369 7818 server_base.cc:1034] running on GCE node
I20260807 06:42:18.245664 7818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:18.245955 7818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:18.247097 7818 hybrid_clock.cc:648] HybridClock initialized: now 1786084938247069 us; error 50 us; skew 500 ppm
Aug 07 06:42:18 dist-test-slave-gkw7 krb5kdc[7802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084938, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:18.254540 7818 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:18.255019 7818 webserver.cc:460] Webserver started at http://127.0.0.1:34045/ using document root <none> and password file <none>
I20260807 06:42:18.255230 7818 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:18.255282 7818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:18.255376 7818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:18.256407 7818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "88d150e61a6e4d28961737eb333a5e77"
format_stamp: "Formatted at 2026-08-07 06:42:18 on dist-test-slave-gkw7"
server_key: "79452cf43239ff035c9c32ca12074bf6"
server_key_iv: "29088ecf5ff7e4750cbb1826c441ef5b"
server_key_version: "encryptionkey@0"
I20260807 06:42:18.256707 7818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "88d150e61a6e4d28961737eb333a5e77"
format_stamp: "Formatted at 2026-08-07 06:42:18 on dist-test-slave-gkw7"
server_key: "79452cf43239ff035c9c32ca12074bf6"
server_key_iv: "29088ecf5ff7e4750cbb1826c441ef5b"
server_key_version: "encryptionkey@0"
I20260807 06:42:18.258122 7818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:18.258891 7844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:18.259086 7818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:18.259197 7818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "88d150e61a6e4d28961737eb333a5e77"
format_stamp: "Formatted at 2026-08-07 06:42:18 on dist-test-slave-gkw7"
server_key: "79452cf43239ff035c9c32ca12074bf6"
server_key_iv: "29088ecf5ff7e4750cbb1826c441ef5b"
server_key_version: "encryptionkey@0"
I20260807 06:42:18.259289 7818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:18.287645 7818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:18.290066 7818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:18.290241 7818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:18.294381 7818 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40207
I20260807 06:42:18.294420 7916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40207 every 8 connection(s)
I20260807 06:42:18.294811 7818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:18.295689 7917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:18.297941 7917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77: Bootstrap starting.
I20260807 06:42:18.298470 7917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:18.298715 7917 log.cc:824] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:18.299405 7917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77: No bootstrap required, opened a new log
I20260807 06:42:18.300168 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 7818
I20260807 06:42:18.300292 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:18.300495 11723 external_mini_cluster.cc:1428] Setting key 536f06de1813d52976b618e0382d61dc
I20260807 06:42:18.301681 7917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d150e61a6e4d28961737eb333a5e77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260807 06:42:18.301868 7917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:18.301913 7917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d150e61a6e4d28961737eb333a5e77, State: Initialized, Role: FOLLOWER
I20260807 06:42:18.302054 7917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d150e61a6e4d28961737eb333a5e77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260807 06:42:18.302142 7917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:18.302181 7917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:18.302229 7917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:18.303257 7917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d150e61a6e4d28961737eb333a5e77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260807 06:42:18.303387 7917 leader_election.cc:302] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d150e61a6e4d28961737eb333a5e77; no voters:
I20260807 06:42:18.303558 7917 leader_election.cc:288] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:18.303612 7922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:18.303764 7922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [term 1 LEADER]: Becoming Leader. State: Replica: 88d150e61a6e4d28961737eb333a5e77, State: Running, Role: LEADER
I20260807 06:42:18.303948 7922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d150e61a6e4d28961737eb333a5e77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260807 06:42:18.303830 7917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:18.304368 7923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88d150e61a6e4d28961737eb333a5e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d150e61a6e4d28961737eb333a5e77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } } }
I20260807 06:42:18.304454 7923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:18.304695 7924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d150e61a6e4d28961737eb333a5e77. Latest consensus state: current_term: 1 leader_uuid: "88d150e61a6e4d28961737eb333a5e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d150e61a6e4d28961737eb333a5e77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } } }
I20260807 06:42:18.304826 7924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:18.304979 7927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:18.305493 7927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:18.307274 7927 catalog_manager.cc:1347] Generated new cluster ID: eba44c31fc3547b1951c6f9cc5db94bf
I20260807 06:42:18.307334 7927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:42:18 dist-test-slave-gkw7 krb5kdc[7802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:42:18.312927 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:18.301051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43003 (local address 127.0.0.1:40207)
0807 06:42:18.301263 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:18.301271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:18.301300 (+ 29us) server_negotiation.cc:407] Connection header received
0807 06:42:18.301655 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:18.301661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:18.301709 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:18.301836 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:18.302710 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:18.303783 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:18.304814 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:18.305426 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:18.305567 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:18.308651 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:18.308654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:18.308655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:18.308665 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:18.310570 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:18.311103 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:18.311106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:18.311107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:18.311243 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:18.311498 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:18.311499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:18.311499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:18.311819 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:18.312512 (+ 693us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:18.312604 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 06:42:18.312712 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260807 06:42:18.313963 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:18.322772 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:18.315567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46519 (local address 127.0.0.1:40207)
0807 06:42:18.315597 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:18.315598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:18.315629 (+ 31us) server_negotiation.cc:407] Connection header received
0807 06:42:18.315700 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:18.315701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:18.315709 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:18.315757 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:18.316346 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:18.317093 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:18.318187 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:18.318513 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:18.318569 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:18.319943 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:18.319944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:18.319944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:18.319950 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:18.321459 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:18.321960 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:18.321961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:18.321962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:18.322017 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:18.322247 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:18.322247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:18.322247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:18.322360 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:18.322460 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:18.322623 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 06:42:18.322686 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260807 06:42:18.328755 7927 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:18.329380 7927 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:18.333283 7927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88d150e61a6e4d28961737eb333a5e77: Generated new TSK 0
I20260807 06:42:18.333468 7927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:18.336954 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 7818
W20260807 06:42:18.351445 7799 connection.cc:535] client connection to 127.0.0.1:40207 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40207): Connection reset by peer (error 104)
2026-08-07T06:42:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:18 dist-test-slave-gkw7 krb5kdc[7959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:18 dist-test-slave-gkw7 krb5kdc[7959](info): set up 2 sockets
Aug 07 06:42:18 dist-test-slave-gkw7 krb5kdc[7959](info): commencing operation
krb5kdc: starting...
W20260807 06:42:19.641798 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:19 dist-test-slave-gkw7 krb5kdc[7959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:19.766819 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:19.848780 7979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:19.849030 7979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:19.849068 7979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:19.850222 7979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:19.850291 7979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:19.850315 7979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:19.850337 7979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:19.850366 7979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:19.851846 7979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41881
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:19.852123 7979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:19.852349 7979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:19.857342 7989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:19.857362 7987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:19.857348 7986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:19.857522 7979 server_base.cc:1034] running on GCE node
I20260807 06:42:19.857810 7979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:19.858143 7979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:19.859356 7979 hybrid_clock.cc:648] HybridClock initialized: now 1786084939859330 us; error 37 us; skew 500 ppm
Aug 07 06:42:19 dist-test-slave-gkw7 krb5kdc[7959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084939, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:19.862211 7979 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:19.862695 7979 webserver.cc:460] Webserver started at http://127.0.0.1:34325/ using document root <none> and password file <none>
I20260807 06:42:19.862941 7979 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:19.863015 7979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:19.863229 7979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:19.864341 7979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "fd90b2711a4b47d797b111cce36097ec"
format_stamp: "Formatted at 2026-08-07 06:42:19 on dist-test-slave-gkw7"
server_key: "92b1c7206217c7784702b6d0bbbcf0d6"
server_key_iv: "2a2e525a945edc4e460d36f85b0dbbe2"
server_key_version: "encryptionkey@0"
I20260807 06:42:19.864753 7979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "fd90b2711a4b47d797b111cce36097ec"
format_stamp: "Formatted at 2026-08-07 06:42:19 on dist-test-slave-gkw7"
server_key: "92b1c7206217c7784702b6d0bbbcf0d6"
server_key_iv: "2a2e525a945edc4e460d36f85b0dbbe2"
server_key_version: "encryptionkey@0"
I20260807 06:42:19.866317 7979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:19.867189 7999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:19.867422 7979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:19.867540 7979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "fd90b2711a4b47d797b111cce36097ec"
format_stamp: "Formatted at 2026-08-07 06:42:19 on dist-test-slave-gkw7"
server_key: "92b1c7206217c7784702b6d0bbbcf0d6"
server_key_iv: "2a2e525a945edc4e460d36f85b0dbbe2"
server_key_version: "encryptionkey@0"
I20260807 06:42:19.867614 7979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:19.875226 7979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:19.877003 7979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:19.877163 7979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:19.881062 7979 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41881
I20260807 06:42:19.881115 8075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41881 every 8 connection(s)
I20260807 06:42:19.881506 7979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:19.881695 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 7979
I20260807 06:42:19.881841 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:19.882064 11723 external_mini_cluster.cc:1428] Setting key b89bed0a483ded526d289cfa9196dafc
I20260807 06:42:19.882668 8077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:19.885802 8077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec: Bootstrap starting.
I20260807 06:42:19.886507 8077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:19.886854 8077 log.cc:824] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:19.887621 8077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec: No bootstrap required, opened a new log
I20260807 06:42:19.887647 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:19.882691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33715 (local address 127.0.0.1:41881)
0807 06:42:19.882884 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:19.882891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:19.882916 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:42:19.883327 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:19.883333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:19.883376 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:19.883487 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:19.883751 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:19.884625 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:19.885779 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:19.886231 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:19.886311 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:19.887027 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:19.887031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:19.887032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:19.887207 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:19.887236 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:19.887247 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:19.887348 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:19.887433 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 06:42:19.887464 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260807 06:42:19.888515 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:19.890022 8077 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90b2711a4b47d797b111cce36097ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } }
I20260807 06:42:19.890164 8077 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:19.890218 8077 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd90b2711a4b47d797b111cce36097ec, State: Initialized, Role: FOLLOWER
I20260807 06:42:19.890336 8077 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90b2711a4b47d797b111cce36097ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } }
I20260807 06:42:19.890432 8077 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:19.890465 8077 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:19.890512 8077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:19.891083 8077 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90b2711a4b47d797b111cce36097ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } }
I20260807 06:42:19.891191 8077 leader_election.cc:302] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd90b2711a4b47d797b111cce36097ec; no voters:
I20260807 06:42:19.891341 8077 leader_election.cc:288] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:19.891373 8088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:19.891549 8088 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [term 1 LEADER]: Becoming Leader. State: Replica: fd90b2711a4b47d797b111cce36097ec, State: Running, Role: LEADER
I20260807 06:42:19.891577 8077 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:19.891713 8088 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90b2711a4b47d797b111cce36097ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } }
I20260807 06:42:19.892056 8088 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd90b2711a4b47d797b111cce36097ec. Latest consensus state: current_term: 1 leader_uuid: "fd90b2711a4b47d797b111cce36097ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90b2711a4b47d797b111cce36097ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } } }
I20260807 06:42:19.892179 8088 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:19.892067 8089 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd90b2711a4b47d797b111cce36097ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90b2711a4b47d797b111cce36097ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } } }
I20260807 06:42:19.892383 8089 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:19.893427 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:19.889939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56603 (local address 127.0.0.1:41881)
0807 06:42:19.889992 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:19.889995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:19.890063 (+ 68us) server_negotiation.cc:407] Connection header received
0807 06:42:19.890146 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:19.890147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:19.890157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:19.890211 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:19.890412 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:19.890938 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:19.891979 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:19.892259 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:19.892330 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:19.893068 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:19.893069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:19.893069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:19.893074 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:19.893087 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:19.893090 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:19.893154 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:19.893286 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 06:42:19.893300 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:42:19.893359 8097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:19.894004 8097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:19.895328 8097 catalog_manager.cc:1347] Generated new cluster ID: ae2115c2334c4c4ba49ad5bec39b6111
I20260807 06:42:19.895402 8097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:19.910630 8097 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:19.911440 8097 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:19.915282 8097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd90b2711a4b47d797b111cce36097ec: Generated new TSK 0
I20260807 06:42:19.915480 8097 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:19.921856 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 7979
W20260807 06:42:19.935549 7957 connection.cc:535] client connection to 127.0.0.1:41881 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41881): Connection reset by peer (error 104)
2026-08-07T06:42:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:19 dist-test-slave-gkw7 krb5kdc[8117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:19 dist-test-slave-gkw7 krb5kdc[8117](info): set up 2 sockets
Aug 07 06:42:19 dist-test-slave-gkw7 krb5kdc[8117](info): commencing operation
krb5kdc: starting...
W20260807 06:42:21.206540 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:21 dist-test-slave-gkw7 krb5kdc[8117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:21.337867 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:21.421029 8136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:21.421253 8136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:21.421277 8136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:21.422394 8136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:21.422434 8136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:21.422449 8136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:21.422462 8136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:21.422474 8136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:21.423914 8136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42417
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:21.424120 8136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:21.424300 8136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:21.429237 8145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:21.429255 8146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:21.429399 8148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:21.429705 8136 server_base.cc:1034] running on GCE node
I20260807 06:42:21.429986 8136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:21.430284 8136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:21.431453 8136 hybrid_clock.cc:648] HybridClock initialized: now 1786084941431446 us; error 35 us; skew 500 ppm
Aug 07 06:42:21 dist-test-slave-gkw7 krb5kdc[8117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084941, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:21.434091 8136 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:21.434545 8136 webserver.cc:460] Webserver started at http://127.0.0.1:43191/ using document root <none> and password file <none>
I20260807 06:42:21.434815 8136 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:21.434887 8136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:21.435039 8136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:21.436152 8136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "132daf2b67774dcd8ad7814be7a7db7d"
format_stamp: "Formatted at 2026-08-07 06:42:21 on dist-test-slave-gkw7"
server_key: "f607e5f087ab854e8b923e18a4b64deb"
server_key_iv: "0c4f3e40aaf7090b41465e2fb22ef3f4"
server_key_version: "encryptionkey@0"
I20260807 06:42:21.436527 8136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "132daf2b67774dcd8ad7814be7a7db7d"
format_stamp: "Formatted at 2026-08-07 06:42:21 on dist-test-slave-gkw7"
server_key: "f607e5f087ab854e8b923e18a4b64deb"
server_key_iv: "0c4f3e40aaf7090b41465e2fb22ef3f4"
server_key_version: "encryptionkey@0"
I20260807 06:42:21.437966 8136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:21.438838 8157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:21.439043 8136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:21.439153 8136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "132daf2b67774dcd8ad7814be7a7db7d"
format_stamp: "Formatted at 2026-08-07 06:42:21 on dist-test-slave-gkw7"
server_key: "f607e5f087ab854e8b923e18a4b64deb"
server_key_iv: "0c4f3e40aaf7090b41465e2fb22ef3f4"
server_key_version: "encryptionkey@0"
I20260807 06:42:21.439231 8136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:21.442924 8136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:21.444921 8136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:21.445089 8136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:21.449234 8136 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42417
I20260807 06:42:21.449275 8227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42417 every 8 connection(s)
I20260807 06:42:21.449702 8136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:21.450595 8229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:21.452754 8229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d: Bootstrap starting.
I20260807 06:42:21.453293 8229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:21.453516 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 8136
I20260807 06:42:21.453676 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:21.453661 8229 log.cc:824] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:21.453867 11723 external_mini_cluster.cc:1428] Setting key dc2dcfdaad81af64a1b814328e9c67c1
I20260807 06:42:21.454535 8229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d: No bootstrap required, opened a new log
I20260807 06:42:21.456537 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:21.454393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40689 (local address 127.0.0.1:42417)
0807 06:42:21.454595 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:21.454605 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:21.454642 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:42:21.455048 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:21.455054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:21.455079 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:21.455236 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:21.455947 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:21.455951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:21.455951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:21.455955 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:21.456007 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:21.456018 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:21.456106 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:21.456164 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 06:42:21.456205 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260807 06:42:21.456537 8229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132daf2b67774dcd8ad7814be7a7db7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } }
I20260807 06:42:21.456739 8229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:21.456781 8229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 132daf2b67774dcd8ad7814be7a7db7d, State: Initialized, Role: FOLLOWER
I20260807 06:42:21.456909 8229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132daf2b67774dcd8ad7814be7a7db7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } }
I20260807 06:42:21.457005 8229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:21.457051 8229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:21.457104 8229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:21.457648 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:21.457875 8229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132daf2b67774dcd8ad7814be7a7db7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } }
I20260807 06:42:21.457995 8229 leader_election.cc:302] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 132daf2b67774dcd8ad7814be7a7db7d; no voters:
I20260807 06:42:21.458178 8229 leader_election.cc:288] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:21.458220 8236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:21.458411 8236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [term 1 LEADER]: Becoming Leader. State: Replica: 132daf2b67774dcd8ad7814be7a7db7d, State: Running, Role: LEADER
I20260807 06:42:21.458461 8229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:21.458576 8236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132daf2b67774dcd8ad7814be7a7db7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } }
I20260807 06:42:21.458945 8238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "132daf2b67774dcd8ad7814be7a7db7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132daf2b67774dcd8ad7814be7a7db7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } } }
I20260807 06:42:21.459043 8238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:21.459282 8252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:21.459930 8252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:21.460511 8237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 132daf2b67774dcd8ad7814be7a7db7d. Latest consensus state: current_term: 1 leader_uuid: "132daf2b67774dcd8ad7814be7a7db7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132daf2b67774dcd8ad7814be7a7db7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } } }
I20260807 06:42:21.460687 8237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:21.460824 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:21.459382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41511 (local address 127.0.0.1:42417)
0807 06:42:21.459420 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:21.459422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:21.459682 (+ 260us) server_negotiation.cc:407] Connection header received
0807 06:42:21.459783 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:21.459784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:21.459790 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:21.459857 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:21.460452 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:21.460453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:21.460454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:21.460458 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:21.460471 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:21.460475 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:21.460530 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:21.460742 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 06:42:21.460757 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:21.461760 8252 catalog_manager.cc:1347] Generated new cluster ID: 9de42780952a46f7aa4da0f6c553d9a7
I20260807 06:42:21.461825 8252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:21.468556 8252 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:21.468747 8252 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:21.472213 8252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 132daf2b67774dcd8ad7814be7a7db7d: Generated new TSK 0
I20260807 06:42:21.472409 8252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:21.474112 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 8136
2026-08-07T06:42:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1555 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:21 dist-test-slave-gkw7 krb5kdc[8269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:21 dist-test-slave-gkw7 krb5kdc[8269](info): set up 2 sockets
Aug 07 06:42:21 dist-test-slave-gkw7 krb5kdc[8269](info): commencing operation
krb5kdc: starting...
W20260807 06:42:22.734719 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:22 dist-test-slave-gkw7 krb5kdc[8269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:22.866699 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:22.950994 8285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:22.951257 8285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:22.951296 8285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:22.952454 8285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:22.952525 8285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:22.952553 8285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:22.952584 8285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:22.952608 8285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:22.954205 8285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:22.954490 8285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:22.954738 8285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:22.960016 8295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:22.960126 8285 server_base.cc:1034] running on GCE node
W20260807 06:42:22.960005 8291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:22.960008 8293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:22.960464 8285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:22.960775 8285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:22.961948 8285 hybrid_clock.cc:648] HybridClock initialized: now 1786084942961945 us; error 42 us; skew 500 ppm
Aug 07 06:42:22 dist-test-slave-gkw7 krb5kdc[8269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084942, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:22.964702 8285 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:22.965265 8285 webserver.cc:460] Webserver started at http://127.0.0.1:36523/ using document root <none> and password file <none>
I20260807 06:42:22.965583 8285 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:22.965669 8285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:22.965833 8285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:22.966987 8285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "3645c6db55b442efaf281a28da1e0201"
format_stamp: "Formatted at 2026-08-07 06:42:22 on dist-test-slave-gkw7"
server_key: "8304a9689f558e7942e7f3c80d71b73b"
server_key_iv: "9e7b75894d80883b89d6b2570da3254f"
server_key_version: "encryptionkey@0"
I20260807 06:42:22.967375 8285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "3645c6db55b442efaf281a28da1e0201"
format_stamp: "Formatted at 2026-08-07 06:42:22 on dist-test-slave-gkw7"
server_key: "8304a9689f558e7942e7f3c80d71b73b"
server_key_iv: "9e7b75894d80883b89d6b2570da3254f"
server_key_version: "encryptionkey@0"
I20260807 06:42:22.968815 8285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:22.969743 8307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:22.970001 8285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:22.970125 8285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "3645c6db55b442efaf281a28da1e0201"
format_stamp: "Formatted at 2026-08-07 06:42:22 on dist-test-slave-gkw7"
server_key: "8304a9689f558e7942e7f3c80d71b73b"
server_key_iv: "9e7b75894d80883b89d6b2570da3254f"
server_key_version: "encryptionkey@0"
I20260807 06:42:22.970208 8285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:22.988601 8285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:22.990942 8285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:22.991125 8285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:22.995731 8285 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41313
I20260807 06:42:22.995797 8374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41313 every 8 connection(s)
I20260807 06:42:22.996176 8285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:22.997129 8376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:22.999670 8376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201: Bootstrap starting.
I20260807 06:42:23.000321 8376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:23.000650 8376 log.cc:824] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:23.001466 8376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201: No bootstrap required, opened a new log
I20260807 06:42:23.002337 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 8285
I20260807 06:42:23.002453 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:23.002642 11723 external_mini_cluster.cc:1428] Setting key a92e8342b57fa45368cdd9e2275b9d11
I20260807 06:42:23.003760 8376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3645c6db55b442efaf281a28da1e0201" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } }
I20260807 06:42:23.003942 8376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:23.003988 8376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3645c6db55b442efaf281a28da1e0201, State: Initialized, Role: FOLLOWER
I20260807 06:42:23.004148 8376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3645c6db55b442efaf281a28da1e0201" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } }
I20260807 06:42:23.004249 8376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:23.004285 8376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:23.004343 8376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:23.005378 8376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3645c6db55b442efaf281a28da1e0201" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } }
I20260807 06:42:23.005496 8376 leader_election.cc:302] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3645c6db55b442efaf281a28da1e0201; no voters:
I20260807 06:42:23.005702 8376 leader_election.cc:288] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:23.005757 8381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:23.005929 8381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [term 1 LEADER]: Becoming Leader. State: Replica: 3645c6db55b442efaf281a28da1e0201, State: Running, Role: LEADER
I20260807 06:42:23.006034 8376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:23.006078 8381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3645c6db55b442efaf281a28da1e0201" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } }
I20260807 06:42:23.006493 8381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3645c6db55b442efaf281a28da1e0201. Latest consensus state: current_term: 1 leader_uuid: "3645c6db55b442efaf281a28da1e0201" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3645c6db55b442efaf281a28da1e0201" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } } }
I20260807 06:42:23.006506 8384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3645c6db55b442efaf281a28da1e0201" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3645c6db55b442efaf281a28da1e0201" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } } }
I20260807 06:42:23.006657 8384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:23.006950 8381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:23.007339 8392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:23.007886 8392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:23.009378 8392 catalog_manager.cc:1347] Generated new cluster ID: 412be7f869544835937c8bb76558e62c
I20260807 06:42:23.009436 8392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:42:23 dist-test-slave-gkw7 krb5kdc[8269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:42:23.014183 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:23.003194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41425 (local address 127.0.0.1:41313)
0807 06:42:23.003396 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:23.003405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:23.003431 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:42:23.003729 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:23.003735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:23.003782 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:23.003899 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:23.004770 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:23.005956 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:23.007091 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:23.007800 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:23.007916 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:23.010676 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:23.010680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:23.010681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:23.010694 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:23.012506 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:23.012901 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:23.012904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:23.012905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:23.013032 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:23.013272 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:23.013273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:23.013273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:23.013567 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:23.013759 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:23.013832 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 06:42:23.013989 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260807 06:42:23.015311 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:23.023717 8392 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:23.024137 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:23.016765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51101 (local address 127.0.0.1:41313)
0807 06:42:23.016805 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:23.016807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:23.017836 (+ 1029us) server_negotiation.cc:407] Connection header received
0807 06:42:23.017914 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:23.017915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:23.017927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:23.017987 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:23.018536 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:23.019024 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:23.019776 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:23.020012 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:23.020108 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:23.021343 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:23.021344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:23.021345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:23.021351 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:23.022816 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:23.023203 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:23.023205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:23.023205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:23.023265 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:23.023474 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:23.023475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:23.023475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:23.023621 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:23.023732 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:23.023947 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 06:42:23.024026 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:42:23.024494 8392 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:23.033030 8392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3645c6db55b442efaf281a28da1e0201: Generated new TSK 0
I20260807 06:42:23.033269 8392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:23.041237 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 8285
2026-08-07T06:42:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1564 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:23 dist-test-slave-gkw7 krb5kdc[8415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:23 dist-test-slave-gkw7 krb5kdc[8415](info): set up 2 sockets
Aug 07 06:42:23 dist-test-slave-gkw7 krb5kdc[8415](info): commencing operation
krb5kdc: starting...
W20260807 06:42:24.364574 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:24 dist-test-slave-gkw7 krb5kdc[8415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:24.490649 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:24.573014 8437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:24.573226 8437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:24.573248 8437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:24.574345 8437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:24.574383 8437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:24.574396 8437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:24.574409 8437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:24.574421 8437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:24.575822 8437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:24.576012 8437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:24.576177 8437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:24.581215 8443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:24.581239 8447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:24.581346 8442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:24.581640 8437 server_base.cc:1034] running on GCE node
I20260807 06:42:24.581919 8437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:24.582211 8437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:24.583410 8437 hybrid_clock.cc:648] HybridClock initialized: now 1786084944583395 us; error 30 us; skew 500 ppm
Aug 07 06:42:24 dist-test-slave-gkw7 krb5kdc[8415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084944, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:24.590576 8437 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:24.591130 8437 webserver.cc:460] Webserver started at http://127.0.0.1:44375/ using document root <none> and password file <none>
I20260807 06:42:24.591619 8437 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:24.591725 8437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:24.591910 8437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:24.593116 8437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "2ab568b7b0a4422d93252c2a2a2b2b41"
format_stamp: "Formatted at 2026-08-07 06:42:24 on dist-test-slave-gkw7"
server_key: "39cabe36d50c7cb62d1e0813c9548027"
server_key_iv: "048eaa030dfabae10e5aecdb53218639"
server_key_version: "encryptionkey@0"
I20260807 06:42:24.593626 8437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "2ab568b7b0a4422d93252c2a2a2b2b41"
format_stamp: "Formatted at 2026-08-07 06:42:24 on dist-test-slave-gkw7"
server_key: "39cabe36d50c7cb62d1e0813c9548027"
server_key_iv: "048eaa030dfabae10e5aecdb53218639"
server_key_version: "encryptionkey@0"
I20260807 06:42:24.595119 8437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:24.596019 8456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:24.596247 8437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:24.596365 8437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "2ab568b7b0a4422d93252c2a2a2b2b41"
format_stamp: "Formatted at 2026-08-07 06:42:24 on dist-test-slave-gkw7"
server_key: "39cabe36d50c7cb62d1e0813c9548027"
server_key_iv: "048eaa030dfabae10e5aecdb53218639"
server_key_version: "encryptionkey@0"
I20260807 06:42:24.596454 8437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:24.627009 8437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:24.629796 8437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:24.629984 8437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:24.634851 8437 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41491
I20260807 06:42:24.634888 8533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41491 every 8 connection(s)
I20260807 06:42:24.635291 8437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:24.636293 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 8437
I20260807 06:42:24.636330 8535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:24.636430 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:24.636655 11723 external_mini_cluster.cc:1428] Setting key 13e0941cff26569c07342239e37eaa0d
I20260807 06:42:24.639649 8535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41: Bootstrap starting.
I20260807 06:42:24.640412 8535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:24.640726 8535 log.cc:824] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:24.641634 8535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41: No bootstrap required, opened a new log
I20260807 06:42:24.642364 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:24.637218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51119 (local address 127.0.0.1:41491)
0807 06:42:24.637405 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:24.637412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:24.637438 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:42:24.637951 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:24.637959 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:24.638010 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:24.638130 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:24.638387 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:24.639291 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:24.640397 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:24.640882 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:24.640994 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:24.641632 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:24.641636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:24.641637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:24.641851 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:24.641883 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:24.641894 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:24.641947 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:24.642050 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 06:42:24.642101 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260807 06:42:24.643477 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:24.644117 8535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab568b7b0a4422d93252c2a2a2b2b41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } }
I20260807 06:42:24.644309 8535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:24.644347 8535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab568b7b0a4422d93252c2a2a2b2b41, State: Initialized, Role: FOLLOWER
I20260807 06:42:24.644470 8535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab568b7b0a4422d93252c2a2a2b2b41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } }
I20260807 06:42:24.644573 8535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:24.644621 8535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:24.644677 8535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:24.645457 8535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab568b7b0a4422d93252c2a2a2b2b41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } }
I20260807 06:42:24.645596 8535 leader_election.cc:302] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ab568b7b0a4422d93252c2a2a2b2b41; no voters:
I20260807 06:42:24.645802 8535 leader_election.cc:288] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:24.645849 8551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:24.646024 8551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [term 1 LEADER]: Becoming Leader. State: Replica: 2ab568b7b0a4422d93252c2a2a2b2b41, State: Running, Role: LEADER
I20260807 06:42:24.646103 8535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:24.646157 8551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab568b7b0a4422d93252c2a2a2b2b41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } }
I20260807 06:42:24.646504 8556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ab568b7b0a4422d93252c2a2a2b2b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab568b7b0a4422d93252c2a2a2b2b41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } } }
I20260807 06:42:24.646592 8556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:24.646497 8551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ab568b7b0a4422d93252c2a2a2b2b41. Latest consensus state: current_term: 1 leader_uuid: "2ab568b7b0a4422d93252c2a2a2b2b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab568b7b0a4422d93252c2a2a2b2b41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } } }
I20260807 06:42:24.646884 8551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:24.647171 8561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:24.647737 8561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:24.648669 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:24.645089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45067 (local address 127.0.0.1:41491)
0807 06:42:24.645126 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:24.645128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:24.645186 (+ 58us) server_negotiation.cc:407] Connection header received
0807 06:42:24.645269 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:24.645269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:24.645277 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:24.645322 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:24.645455 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:24.646244 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:24.647259 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:24.647549 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:24.647618 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:24.648334 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:24.648336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:24.648336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:24.648341 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:24.648356 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:24.648360 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:24.648553 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:24.648570 (+ 17us) server_negotiation.cc:299] Negotiation successful
0807 06:42:24.648592 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:42:24.649549 8561 catalog_manager.cc:1347] Generated new cluster ID: 86218927f2f54f108d6b5307de3a2d57
I20260807 06:42:24.649602 8561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:24.657964 8561 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:24.658684 8561 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:24.661895 8561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ab568b7b0a4422d93252c2a2a2b2b41: Generated new TSK 0
I20260807 06:42:24.662079 8561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:24.667986 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 8437
2026-08-07T06:42:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1619 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:24 dist-test-slave-gkw7 krb5kdc[8579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:24 dist-test-slave-gkw7 krb5kdc[8579](info): set up 2 sockets
Aug 07 06:42:24 dist-test-slave-gkw7 krb5kdc[8579](info): commencing operation
krb5kdc: starting...
W20260807 06:42:25.871529 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:25 dist-test-slave-gkw7 krb5kdc[8579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:25.994158 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41089
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:26.077445 8597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:26.077697 8597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:26.077721 8597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:26.078979 8597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:26.079030 8597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:26.079044 8597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:26.079057 8597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:26.079070 8597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:26.080595 8597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:26.080863 8597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:26.081106 8597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:26.086275 8602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:26.086272 8608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:26.086297 8603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:26.086493 8597 server_base.cc:1034] running on GCE node
I20260807 06:42:26.086766 8597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:26.087054 8597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:26.088225 8597 hybrid_clock.cc:648] HybridClock initialized: now 1786084946088215 us; error 42 us; skew 500 ppm
Aug 07 06:42:26 dist-test-slave-gkw7 krb5kdc[8579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084946, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:26.090982 8597 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:26.091490 8597 webserver.cc:460] Webserver started at http://127.0.0.1:39211/ using document root <none> and password file <none>
I20260807 06:42:26.091732 8597 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:26.091804 8597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:26.091969 8597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:26.093065 8597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "53737599a29349c28114afd456ab56dc"
format_stamp: "Formatted at 2026-08-07 06:42:26 on dist-test-slave-gkw7"
server_key: "dd12a0fe2d39d6c5183d5591added4c2"
server_key_iv: "979496702cbd3a693f0a0db0fdfa7fe1"
server_key_version: "encryptionkey@0"
I20260807 06:42:26.093453 8597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "53737599a29349c28114afd456ab56dc"
format_stamp: "Formatted at 2026-08-07 06:42:26 on dist-test-slave-gkw7"
server_key: "dd12a0fe2d39d6c5183d5591added4c2"
server_key_iv: "979496702cbd3a693f0a0db0fdfa7fe1"
server_key_version: "encryptionkey@0"
I20260807 06:42:26.094892 8597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:26.095784 8618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:26.095959 8597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:42:26.096113 8597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "53737599a29349c28114afd456ab56dc"
format_stamp: "Formatted at 2026-08-07 06:42:26 on dist-test-slave-gkw7"
server_key: "dd12a0fe2d39d6c5183d5591added4c2"
server_key_iv: "979496702cbd3a693f0a0db0fdfa7fe1"
server_key_version: "encryptionkey@0"
I20260807 06:42:26.096211 8597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:26.099763 8597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:26.102350 8597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:26.102528 8597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:26.106843 8597 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41089
I20260807 06:42:26.106873 8690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41089 every 8 connection(s)
I20260807 06:42:26.107249 8597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:26.108140 8692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:26.109635 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 8597
I20260807 06:42:26.109767 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:26.109984 11723 external_mini_cluster.cc:1428] Setting key f7388ad40713fcef32177fbb87f4fee8
I20260807 06:42:26.111351 8692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc: Bootstrap starting.
I20260807 06:42:26.112105 8692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:26.112447 8692 log.cc:824] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:26.113168 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:26.110573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56129 (local address 127.0.0.1:41089)
0807 06:42:26.110846 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:26.110856 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:26.110971 (+ 115us) server_negotiation.cc:407] Connection header received
0807 06:42:26.111722 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:26.111731 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:26.111794 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:26.111972 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:26.112726 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:26.112729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:26.112730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:26.112734 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:26.112782 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:26.112793 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:26.112889 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:26.112957 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 06:42:26.112999 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":74,"threads_started":1}
I20260807 06:42:26.113287 8692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc: No bootstrap required, opened a new log
I20260807 06:42:26.114174 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:26.115753 8692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53737599a29349c28114afd456ab56dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260807 06:42:26.115949 8692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:26.116006 8692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53737599a29349c28114afd456ab56dc, State: Initialized, Role: FOLLOWER
I20260807 06:42:26.116143 8692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53737599a29349c28114afd456ab56dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260807 06:42:26.116271 8692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:26.116333 8692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:26.116389 8692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:26.116798 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:26.115643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47457 (local address 127.0.0.1:41089)
0807 06:42:26.115698 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:26.115700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:26.115818 (+ 118us) server_negotiation.cc:407] Connection header received
0807 06:42:26.115874 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:26.115875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:26.115879 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:26.115929 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:26.116556 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:26.116558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:26.116558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:26.116563 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:26.116577 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:26.116581 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:26.116652 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:26.116717 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 06:42:26.116732 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:42:26.117090 8692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53737599a29349c28114afd456ab56dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260807 06:42:26.117200 8692 leader_election.cc:302] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53737599a29349c28114afd456ab56dc; no voters:
I20260807 06:42:26.117369 8692 leader_election.cc:288] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:26.117408 8706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:26.117542 8706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [term 1 LEADER]: Becoming Leader. State: Replica: 53737599a29349c28114afd456ab56dc, State: Running, Role: LEADER
I20260807 06:42:26.117681 8692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:26.117746 8706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53737599a29349c28114afd456ab56dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260807 06:42:26.118136 8707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53737599a29349c28114afd456ab56dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53737599a29349c28114afd456ab56dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } } }
I20260807 06:42:26.118140 8708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53737599a29349c28114afd456ab56dc. Latest consensus state: current_term: 1 leader_uuid: "53737599a29349c28114afd456ab56dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53737599a29349c28114afd456ab56dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } } }
I20260807 06:42:26.118319 8707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:26.118326 8708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:26.118569 8721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:26.119055 8721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:26.120718 8721 catalog_manager.cc:1347] Generated new cluster ID: a8e69aca620b4ca4ad72d4af3abe40c0
I20260807 06:42:26.120786 8721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:26.139792 8721 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:26.139988 8721 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:26.144407 8721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53737599a29349c28114afd456ab56dc: Generated new TSK 0
I20260807 06:42:26.144600 8721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:26.150568 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 8597
2026-08-07T06:42:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:26 dist-test-slave-gkw7 krb5kdc[8739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:26 dist-test-slave-gkw7 krb5kdc[8739](info): set up 2 sockets
Aug 07 06:42:26 dist-test-slave-gkw7 krb5kdc[8739](info): commencing operation
krb5kdc: starting...
W20260807 06:42:27.380232 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:27 dist-test-slave-gkw7 krb5kdc[8739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:27.509719 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43947
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:27.594923 8755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:27.595187 8755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:27.595225 8755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:27.596397 8755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:27.596472 8755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:27.596494 8755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:27.596515 8755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:27.596546 8755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:27.598089 8755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:27.598389 8755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:27.598629 8755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:27.603586 8763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:27.603586 8761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:27.603816 8760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:27.604022 8755 server_base.cc:1034] running on GCE node
I20260807 06:42:27.604303 8755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:27.604549 8755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:27.605690 8755 hybrid_clock.cc:648] HybridClock initialized: now 1786084947605654 us; error 50 us; skew 500 ppm
Aug 07 06:42:27 dist-test-slave-gkw7 krb5kdc[8739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084947, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:27.608261 8755 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:27.608762 8755 webserver.cc:460] Webserver started at http://127.0.0.1:42743/ using document root <none> and password file <none>
I20260807 06:42:27.609021 8755 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:27.609067 8755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:27.609236 8755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:27.610394 8755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "1de3d222d5f64cfc947bb561cb29cab0"
format_stamp: "Formatted at 2026-08-07 06:42:27 on dist-test-slave-gkw7"
server_key: "5f2d5688691a987a9475505c1f13e1ac"
server_key_iv: "81fb288b6e7237b1b00162f714de5c7d"
server_key_version: "encryptionkey@0"
I20260807 06:42:27.610764 8755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "1de3d222d5f64cfc947bb561cb29cab0"
format_stamp: "Formatted at 2026-08-07 06:42:27 on dist-test-slave-gkw7"
server_key: "5f2d5688691a987a9475505c1f13e1ac"
server_key_iv: "81fb288b6e7237b1b00162f714de5c7d"
server_key_version: "encryptionkey@0"
I20260807 06:42:27.612139 8755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:42:27.612919 8775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:27.613101 8755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:42:27.613215 8755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "1de3d222d5f64cfc947bb561cb29cab0"
format_stamp: "Formatted at 2026-08-07 06:42:27 on dist-test-slave-gkw7"
server_key: "5f2d5688691a987a9475505c1f13e1ac"
server_key_iv: "81fb288b6e7237b1b00162f714de5c7d"
server_key_version: "encryptionkey@0"
I20260807 06:42:27.613306 8755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:27.624400 8755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:27.626621 8755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:27.626806 8755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:27.630986 8755 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43947
I20260807 06:42:27.631026 8845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43947 every 8 connection(s)
I20260807 06:42:27.631443 8755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:27.632448 8847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:27.635133 8847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0: Bootstrap starting.
I20260807 06:42:27.635464 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 8755
I20260807 06:42:27.635589 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:27.635826 11723 external_mini_cluster.cc:1428] Setting key 75077ca24330b250be5f7a763539cb86
I20260807 06:42:27.636562 8847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:27.636917 8847 log.cc:824] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:27.637784 8847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0: No bootstrap required, opened a new log
I20260807 06:42:27.639781 8847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de3d222d5f64cfc947bb561cb29cab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260807 06:42:27.639954 8847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:27.640007 8847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de3d222d5f64cfc947bb561cb29cab0, State: Initialized, Role: FOLLOWER
I20260807 06:42:27.640166 8847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de3d222d5f64cfc947bb561cb29cab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260807 06:42:27.640259 8847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:27.640291 8847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:27.640332 8847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:27.641126 8847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de3d222d5f64cfc947bb561cb29cab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260807 06:42:27.641273 8847 leader_election.cc:302] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1de3d222d5f64cfc947bb561cb29cab0; no voters:
I20260807 06:42:27.641479 8847 leader_election.cc:288] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:27.641574 8853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:27.641736 8853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [term 1 LEADER]: Becoming Leader. State: Replica: 1de3d222d5f64cfc947bb561cb29cab0, State: Running, Role: LEADER
I20260807 06:42:27.641857 8847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:27.641919 8853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de3d222d5f64cfc947bb561cb29cab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260807 06:42:27.642374 8853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1de3d222d5f64cfc947bb561cb29cab0. Latest consensus state: current_term: 1 leader_uuid: "1de3d222d5f64cfc947bb561cb29cab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de3d222d5f64cfc947bb561cb29cab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } } }
I20260807 06:42:27.642386 8854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1de3d222d5f64cfc947bb561cb29cab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de3d222d5f64cfc947bb561cb29cab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } } }
I20260807 06:42:27.642534 8854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:27.642796 8853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:27.642982 8865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 06:42:27 dist-test-slave-gkw7 krb5kdc[8739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:42:27.643522 8865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:27.646662 8865 catalog_manager.cc:1347] Generated new cluster ID: 10a4e0c8dd1e458a868bd70b4781a601
I20260807 06:42:27.646719 8865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:27.648183 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:27.636506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:48401 (local address 127.0.0.1:43947)
0807 06:42:27.636807 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:27.636818 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:27.636856 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:42:27.637376 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:27.637385 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:27.637437 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:27.637622 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:27.638390 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:27.639297 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:27.640109 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:27.640813 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:27.640951 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:27.644292 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:27.644296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:27.644296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:27.644310 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:27.646375 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:27.646875 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:27.646878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:27.646879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:27.646986 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:27.647267 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:27.647268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:27.647268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:27.647561 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:27.647703 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:27.647876 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 06:42:27.647977 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":144,"threads_started":1}
I20260807 06:42:27.649183 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:27.658875 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:27.650799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:55983 (local address 127.0.0.1:43947)
0807 06:42:27.650842 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:27.650844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:27.650911 (+ 67us) server_negotiation.cc:407] Connection header received
0807 06:42:27.650990 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:27.650991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:27.651000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:27.651053 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:27.651796 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:27.652546 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:27.653643 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:27.653965 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:27.654031 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:27.655549 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:27.655551 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:27.655551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:27.655558 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:27.657338 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:27.657886 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:27.657888 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:27.657888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:27.657974 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:27.658223 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:27.658224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:27.658224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:27.658359 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:27.658453 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:27.658610 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 06:42:27.658674 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:42:27.667791 8865 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:27.668498 8865 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:27.671877 8865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1de3d222d5f64cfc947bb561cb29cab0: Generated new TSK 0
I20260807 06:42:27.672076 8865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:27.675504 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 8755
W20260807 06:42:27.684057 8737 connection.cc:535] client connection to 127.0.0.1:43947 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43947): Connection reset by peer (error 104)
2026-08-07T06:42:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:27 dist-test-slave-gkw7 krb5kdc[8891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:27 dist-test-slave-gkw7 krb5kdc[8891](info): set up 2 sockets
Aug 07 06:42:27 dist-test-slave-gkw7 krb5kdc[8891](info): commencing operation
krb5kdc: starting...
W20260807 06:42:28.927541 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:29 dist-test-slave-gkw7 krb5kdc[8891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:29.050738 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:29.135962 8910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:29.136185 8910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:29.136209 8910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:29.137316 8910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:29.137352 8910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:29.137367 8910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:29.137378 8910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:29.137390 8910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:29.138935 8910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:29.139146 8910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:29.139315 8910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:29.144299 8919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:29.144243 8921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:29.144281 8910 server_base.cc:1034] running on GCE node
W20260807 06:42:29.144263 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:29.144745 8910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:29.145097 8910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:29.146283 8910 hybrid_clock.cc:648] HybridClock initialized: now 1786084949146268 us; error 45 us; skew 500 ppm
Aug 07 06:42:29 dist-test-slave-gkw7 krb5kdc[8891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:29.148837 8910 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:29.149314 8910 webserver.cc:460] Webserver started at http://127.0.0.1:38153/ using document root <none> and password file <none>
I20260807 06:42:29.149616 8910 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:29.149686 8910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:29.149827 8910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:29.150909 8910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "10dc25d927f043bea66ad5952db4f920"
format_stamp: "Formatted at 2026-08-07 06:42:29 on dist-test-slave-gkw7"
server_key: "48723af6636373ef0037b7f61d5f0bbc"
server_key_iv: "4d79dddfc34eb7a5db5bb96f689f0fd6"
server_key_version: "encryptionkey@0"
I20260807 06:42:29.151295 8910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "10dc25d927f043bea66ad5952db4f920"
format_stamp: "Formatted at 2026-08-07 06:42:29 on dist-test-slave-gkw7"
server_key: "48723af6636373ef0037b7f61d5f0bbc"
server_key_iv: "4d79dddfc34eb7a5db5bb96f689f0fd6"
server_key_version: "encryptionkey@0"
I20260807 06:42:29.152662 8910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:29.153481 8928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:29.153704 8910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:29.153811 8910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "10dc25d927f043bea66ad5952db4f920"
format_stamp: "Formatted at 2026-08-07 06:42:29 on dist-test-slave-gkw7"
server_key: "48723af6636373ef0037b7f61d5f0bbc"
server_key_iv: "4d79dddfc34eb7a5db5bb96f689f0fd6"
server_key_version: "encryptionkey@0"
I20260807 06:42:29.153908 8910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:29.172434 8910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:29.174458 8910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:29.174635 8910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:29.179059 8910 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42127
I20260807 06:42:29.179118 8998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42127 every 8 connection(s)
I20260807 06:42:29.179517 8910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:29.180507 8999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:29.183054 8999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920: Bootstrap starting.
I20260807 06:42:29.183666 8999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:29.183964 8999 log.cc:824] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:29.184677 8999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920: No bootstrap required, opened a new log
I20260807 06:42:29.186517 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 8910
I20260807 06:42:29.186403 8999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10dc25d927f043bea66ad5952db4f920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } }
I20260807 06:42:29.186583 8999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:29.186635 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:29.186623 8999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10dc25d927f043bea66ad5952db4f920, State: Initialized, Role: FOLLOWER
I20260807 06:42:29.186785 8999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10dc25d927f043bea66ad5952db4f920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } }
I20260807 06:42:29.186865 11723 external_mini_cluster.cc:1428] Setting key 625810dc494959c52a1d9ddc37752196
I20260807 06:42:29.186877 8999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:29.186913 8999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:29.186965 8999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:29.188095 8999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10dc25d927f043bea66ad5952db4f920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } }
I20260807 06:42:29.188238 8999 leader_election.cc:302] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10dc25d927f043bea66ad5952db4f920; no voters:
I20260807 06:42:29.188433 8999 leader_election.cc:288] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:29.188486 9005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:29.188663 9005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [term 1 LEADER]: Becoming Leader. State: Replica: 10dc25d927f043bea66ad5952db4f920, State: Running, Role: LEADER
I20260807 06:42:29.188728 8999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:29.188812 9005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10dc25d927f043bea66ad5952db4f920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } }
I20260807 06:42:29.189257 9007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10dc25d927f043bea66ad5952db4f920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10dc25d927f043bea66ad5952db4f920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } } }
I20260807 06:42:29.189330 9007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:29.189509 9008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10dc25d927f043bea66ad5952db4f920. Latest consensus state: current_term: 1 leader_uuid: "10dc25d927f043bea66ad5952db4f920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10dc25d927f043bea66ad5952db4f920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } } }
I20260807 06:42:29.189653 9008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:29.189942 9015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:29.190645 9015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:29.192389 9015 catalog_manager.cc:1347] Generated new cluster ID: a13721befe174585b50831f69cba264c
I20260807 06:42:29.192444 9015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:29.193073 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:29.187398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:59675 (local address 127.0.0.1:42127)
0807 06:42:29.187622 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:29.187638 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:29.187665 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:42:29.187758 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:29.187763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:29.187807 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:29.187916 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:29.188202 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:29.189601 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:29.190724 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:29.191325 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:29.191458 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:29.192262 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:29.192266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:29.192266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:29.192480 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:29.192520 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:29.192535 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:29.192622 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:29.192827 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 06:42:29.192855 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":137,"threads_started":1}
I20260807 06:42:29.194229 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:29.199412 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:29.195786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:36999 (local address 127.0.0.1:42127)
0807 06:42:29.195821 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:29.195823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:29.195943 (+ 120us) server_negotiation.cc:407] Connection header received
0807 06:42:29.196003 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:29.196004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:29.196015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:29.196078 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:29.196212 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:29.196956 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:29.197958 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:29.198289 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:29.198383 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:29.199135 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:29.199137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:29.199137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:29.199142 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:29.199158 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:29.199162 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:29.199233 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:29.199318 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 06:42:29.199332 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:42:29.216841 9015 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:29.217427 9015 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:29.226534 9015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10dc25d927f043bea66ad5952db4f920: Generated new TSK 0
I20260807 06:42:29.226753 9015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:29.237641 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 8910
W20260807 06:42:29.247493 8885 connection.cc:535] client connection to 127.0.0.1:42127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42127): Connection reset by peer (error 104)
2026-08-07T06:42:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1563 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:29 dist-test-slave-gkw7 krb5kdc[9044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:29 dist-test-slave-gkw7 krb5kdc[9044](info): set up 2 sockets
Aug 07 06:42:29 dist-test-slave-gkw7 krb5kdc[9044](info): commencing operation
krb5kdc: starting...
W20260807 06:42:30.459597 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:30 dist-test-slave-gkw7 krb5kdc[9044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:30.586755 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40791
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:30.670819 9059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:30.671021 9059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:30.671043 9059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:30.672181 9059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:30.672226 9059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:30.672241 9059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:30.672253 9059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:30.672266 9059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:30.673784 9059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:30.674081 9059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:30.674312 9059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:42:30.679415 9059 server_base.cc:1034] running on GCE node
W20260807 06:42:30.679373 9070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:30.679394 9068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:30.679378 9067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:30.679898 9059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:30.680186 9059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:30.681378 9059 hybrid_clock.cc:648] HybridClock initialized: now 1786084950681361 us; error 28 us; skew 500 ppm
Aug 07 06:42:30 dist-test-slave-gkw7 krb5kdc[9044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:30.691255 9059 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:30.691785 9059 webserver.cc:460] Webserver started at http://127.0.0.1:37045/ using document root <none> and password file <none>
I20260807 06:42:30.692070 9059 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:30.692149 9059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:30.692305 9059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:30.693478 9059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "1f7d110e2d88482baf5d6b6104488980"
format_stamp: "Formatted at 2026-08-07 06:42:30 on dist-test-slave-gkw7"
server_key: "228d13a5d8148c32870bdfe16450fba2"
server_key_iv: "7909af33f77d320052d9db62318d09a4"
server_key_version: "encryptionkey@0"
I20260807 06:42:30.693972 9059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "1f7d110e2d88482baf5d6b6104488980"
format_stamp: "Formatted at 2026-08-07 06:42:30 on dist-test-slave-gkw7"
server_key: "228d13a5d8148c32870bdfe16450fba2"
server_key_iv: "7909af33f77d320052d9db62318d09a4"
server_key_version: "encryptionkey@0"
I20260807 06:42:30.695506 9059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:42:30.696355 9079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:30.696545 9059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:30.696674 9059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "1f7d110e2d88482baf5d6b6104488980"
format_stamp: "Formatted at 2026-08-07 06:42:30 on dist-test-slave-gkw7"
server_key: "228d13a5d8148c32870bdfe16450fba2"
server_key_iv: "7909af33f77d320052d9db62318d09a4"
server_key_version: "encryptionkey@0"
I20260807 06:42:30.696763 9059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:30.700527 9059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:30.702684 9059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:30.702855 9059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:30.707288 9059 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40791
I20260807 06:42:30.707319 9151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40791 every 8 connection(s)
I20260807 06:42:30.707693 9059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:30.708712 9154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:30.711107 9154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980: Bootstrap starting.
I20260807 06:42:30.711724 9154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:30.712040 9154 log.cc:824] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:30.712181 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 9059
I20260807 06:42:30.712313 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:30.712558 11723 external_mini_cluster.cc:1428] Setting key 08a7398ff23ea618ad21f5cb4e7ad188
I20260807 06:42:30.712929 9154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980: No bootstrap required, opened a new log
I20260807 06:42:30.715335 9154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7d110e2d88482baf5d6b6104488980" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40791 } }
I20260807 06:42:30.715518 9154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:30.715564 9154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7d110e2d88482baf5d6b6104488980, State: Initialized, Role: FOLLOWER
I20260807 06:42:30.715721 9154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7d110e2d88482baf5d6b6104488980" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40791 } }
I20260807 06:42:30.715839 9154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:30.715874 9154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:30.715922 9154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:30.716073 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:30.713100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:56701 (local address 127.0.0.1:40791)
0807 06:42:30.713308 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:30.713341 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:30.713375 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:42:30.714458 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:30.714468 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:30.714504 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:30.714677 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:30.715387 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:30.715391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:30.715391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:30.715396 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:30.715439 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:30.715452 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:30.715525 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:30.715690 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 06:42:30.715846 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
I20260807 06:42:30.716964 9154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7d110e2d88482baf5d6b6104488980" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40791 } }
I20260807 06:42:30.717072 9154 leader_election.cc:302] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f7d110e2d88482baf5d6b6104488980; no voters:
I20260807 06:42:30.717276 9154 leader_election.cc:288] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:30.717319 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:30.717355 9162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:30.717509 9162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [term 1 LEADER]: Becoming Leader. State: Replica: 1f7d110e2d88482baf5d6b6104488980, State: Running, Role: LEADER
I20260807 06:42:30.717566 9154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:30.717698 9162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7d110e2d88482baf5d6b6104488980" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40791 } }
I20260807 06:42:30.718007 9162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f7d110e2d88482baf5d6b6104488980. Latest consensus state: current_term: 1 leader_uuid: "1f7d110e2d88482baf5d6b6104488980" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7d110e2d88482baf5d6b6104488980" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40791 } } }
I20260807 06:42:30.718098 9162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:30.718024 9165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f7d110e2d88482baf5d6b6104488980" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7d110e2d88482baf5d6b6104488980" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40791 } } }
I20260807 06:42:30.718282 9165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:30.719376 9192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:30.720027 9192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:30.720494 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:30.719118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:35109 (local address 127.0.0.1:40791)
0807 06:42:30.719158 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:30.719160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:30.719327 (+ 167us) server_negotiation.cc:407] Connection header received
0807 06:42:30.719448 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:30.719449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:30.719454 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:30.719522 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:30.720194 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:30.720195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:30.720196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:30.720201 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:30.720215 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:30.720219 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:30.720278 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:30.720424 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 06:42:30.720437 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:30.721675 9192 catalog_manager.cc:1347] Generated new cluster ID: 56bce1448da1409bbcdfd0fb6fb9e2ed
I20260807 06:42:30.721747 9192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:30.727553 9192 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:30.727720 9192 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:30.730158 9192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f7d110e2d88482baf5d6b6104488980: Generated new TSK 0
I20260807 06:42:30.730333 9192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:30.732632 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 9059
2026-08-07T06:42:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:30 dist-test-slave-gkw7 krb5kdc[9197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:30 dist-test-slave-gkw7 krb5kdc[9197](info): set up 2 sockets
Aug 07 06:42:30 dist-test-slave-gkw7 krb5kdc[9197](info): commencing operation
krb5kdc: starting...
W20260807 06:42:31.938606 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:32 dist-test-slave-gkw7 krb5kdc[9197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:32.069027 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:32.166932 9213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:32.167245 9213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:32.167282 9213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:32.169046 9213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:32.169090 9213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:32.169106 9213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:32.169167 9213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:32.169199 9213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:32.171267 9213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38561
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:32.171550 9213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:32.171847 9213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:32.178632 9220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:32.178618 9221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:32.178720 9224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:32.178916 9213 server_base.cc:1034] running on GCE node
I20260807 06:42:32.179207 9213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:32.179571 9213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:32.180750 9213 hybrid_clock.cc:648] HybridClock initialized: now 1786084952180735 us; error 27 us; skew 500 ppm
Aug 07 06:42:32 dist-test-slave-gkw7 krb5kdc[9197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084952, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:32.183781 9213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:32.184432 9213 webserver.cc:460] Webserver started at http://127.0.0.1:42563/ using document root <none> and password file <none>
I20260807 06:42:32.184732 9213 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:32.184813 9213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:32.184973 9213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:32.186609 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "984952a2458b4aabab7d6fc3bc72bb9c"
format_stamp: "Formatted at 2026-08-07 06:42:32 on dist-test-slave-gkw7"
server_key: "b766c711d5f39358954c2a85187a29c6"
server_key_iv: "8b3eb7034f5180ba1b1418cf69a612f8"
server_key_version: "encryptionkey@0"
I20260807 06:42:32.187109 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "984952a2458b4aabab7d6fc3bc72bb9c"
format_stamp: "Formatted at 2026-08-07 06:42:32 on dist-test-slave-gkw7"
server_key: "b766c711d5f39358954c2a85187a29c6"
server_key_iv: "8b3eb7034f5180ba1b1418cf69a612f8"
server_key_version: "encryptionkey@0"
I20260807 06:42:32.189105 9213 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260807 06:42:32.190225 9231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:32.190444 9213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:32.190559 9213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "984952a2458b4aabab7d6fc3bc72bb9c"
format_stamp: "Formatted at 2026-08-07 06:42:32 on dist-test-slave-gkw7"
server_key: "b766c711d5f39358954c2a85187a29c6"
server_key_iv: "8b3eb7034f5180ba1b1418cf69a612f8"
server_key_version: "encryptionkey@0"
I20260807 06:42:32.190650 9213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:32.217665 9213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:32.219749 9213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:32.219877 9213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:32.224263 9213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38561
I20260807 06:42:32.224342 9305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38561 every 8 connection(s)
I20260807 06:42:32.224749 9213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:32.225270 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 9213
I20260807 06:42:32.225404 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:32.225687 11723 external_mini_cluster.cc:1428] Setting key 9d4ced3bffd9b972bf6600af325003ec
I20260807 06:42:32.226012 9306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:32.229337 9306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c: Bootstrap starting.
I20260807 06:42:32.230173 9306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:32.230502 9306 log.cc:824] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:32.231360 9306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c: No bootstrap required, opened a new log
Aug 07 06:42:32 dist-test-slave-gkw7 krb5kdc[9197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:42:32.234083 9306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984952a2458b4aabab7d6fc3bc72bb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } }
I20260807 06:42:32.234277 9306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:32.234323 9306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984952a2458b4aabab7d6fc3bc72bb9c, State: Initialized, Role: FOLLOWER
I20260807 06:42:32.234462 9306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984952a2458b4aabab7d6fc3bc72bb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } }
I20260807 06:42:32.234560 9306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:32.234612 9306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:32.234678 9306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:32.235468 9306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984952a2458b4aabab7d6fc3bc72bb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } }
I20260807 06:42:32.235594 9306 leader_election.cc:302] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 984952a2458b4aabab7d6fc3bc72bb9c; no voters:
I20260807 06:42:32.235762 9306 leader_election.cc:288] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:32.235838 9312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:32.236009 9312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [term 1 LEADER]: Becoming Leader. State: Replica: 984952a2458b4aabab7d6fc3bc72bb9c, State: Running, Role: LEADER
I20260807 06:42:32.236085 9306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:32.236095 9312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984952a2458b4aabab7d6fc3bc72bb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } }
I20260807 06:42:32.236496 9315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 984952a2458b4aabab7d6fc3bc72bb9c. Latest consensus state: current_term: 1 leader_uuid: "984952a2458b4aabab7d6fc3bc72bb9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984952a2458b4aabab7d6fc3bc72bb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } } }
I20260807 06:42:32.236614 9315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:32.237350 9313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "984952a2458b4aabab7d6fc3bc72bb9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984952a2458b4aabab7d6fc3bc72bb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } } }
I20260807 06:42:32.237447 9313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c [sys.catalog]: This master's current role is: LEADER
W20260807 06:42:32.237994 9336 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:42:32.238075 9336 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:42:32.238155 9321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:32.238340 9321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:32.238483 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:32.226341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:38229 (local address 127.0.0.1:38561)
0807 06:42:32.226509 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:32.226517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:32.226554 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:42:32.227428 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:32.227437 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:32.227490 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:32.227615 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:32.228542 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:32.229421 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:32.230605 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:32.231088 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:32.231180 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:32.234011 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:32.234016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:32.234016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:32.234032 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:32.236237 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:32.236871 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:32.236874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:32.236874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:32.236985 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:32.237242 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:32.237244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:32.237257 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:32.237636 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:32.237801 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:32.238133 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 06:42:32.238235 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260807 06:42:32.239594 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:32.240201 9321 catalog_manager.cc:1347] Generated new cluster ID: 0aaaf3ef33a34159b1952bc90cd684af
I20260807 06:42:32.240257 9321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:32.247309 9321 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:32.247967 9321 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:32.249739 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:32.241152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:36465 (local address 127.0.0.1:38561)
0807 06:42:32.241197 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:32.241198 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:32.241736 (+ 538us) server_negotiation.cc:407] Connection header received
0807 06:42:32.241805 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:32.241806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:32.241818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:32.241885 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:32.242523 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:32.243591 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:32.244591 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:32.244930 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:32.245007 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:32.246540 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:32.246541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:32.246541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:32.246549 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:32.248380 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:32.248827 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:32.248828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:32.248829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:32.248893 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:32.249149 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:32.249150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:32.249150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:32.249280 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:32.249386 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:32.249581 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 06:42:32.249649 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:42:32.261652 9321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 984952a2458b4aabab7d6fc3bc72bb9c: Generated new TSK 0
I20260807 06:42:32.261883 9321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:32.267644 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 9213
W20260807 06:42:32.277946 9195 connection.cc:535] client connection to 127.0.0.1:38561 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38561): Connection reset by peer (error 104)
2026-08-07T06:42:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1534 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:32 dist-test-slave-gkw7 krb5kdc[9347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:32 dist-test-slave-gkw7 krb5kdc[9347](info): set up 2 sockets
Aug 07 06:42:32 dist-test-slave-gkw7 krb5kdc[9347](info): commencing operation
krb5kdc: starting...
W20260807 06:42:33.480876 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:33 dist-test-slave-gkw7 krb5kdc[9347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:33.606206 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36077
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:33.688639 9366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:33.688901 9366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:33.688941 9366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:33.690184 9366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:33.690254 9366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:33.690279 9366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:33.690310 9366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:33.690337 9366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:33.691936 9366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36077
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:33.692224 9366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:33.692464 9366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:33.697412 9373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:33.697424 9371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:33.697424 9380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:33.697840 9366 server_base.cc:1034] running on GCE node
I20260807 06:42:33.698084 9366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:33.698388 9366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:33.699538 9366 hybrid_clock.cc:648] HybridClock initialized: now 1786084953699528 us; error 35 us; skew 500 ppm
Aug 07 06:42:33 dist-test-slave-gkw7 krb5kdc[9347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:33.702164 9366 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:33.702615 9366 webserver.cc:460] Webserver started at http://127.0.0.1:46097/ using document root <none> and password file <none>
I20260807 06:42:33.702821 9366 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:33.702862 9366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:33.702952 9366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:33.703965 9366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "2ae969c920554283bd0920ab9ea1bd4e"
format_stamp: "Formatted at 2026-08-07 06:42:33 on dist-test-slave-gkw7"
server_key: "72e9fdfa9611df8875347d7dff3714dc"
server_key_iv: "4d4882b8e03deaeefccaa1eeac2901bb"
server_key_version: "encryptionkey@0"
I20260807 06:42:33.704263 9366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "2ae969c920554283bd0920ab9ea1bd4e"
format_stamp: "Formatted at 2026-08-07 06:42:33 on dist-test-slave-gkw7"
server_key: "72e9fdfa9611df8875347d7dff3714dc"
server_key_iv: "4d4882b8e03deaeefccaa1eeac2901bb"
server_key_version: "encryptionkey@0"
I20260807 06:42:33.705592 9366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:42:33.706308 9390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:33.706487 9366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:42:33.706598 9366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "2ae969c920554283bd0920ab9ea1bd4e"
format_stamp: "Formatted at 2026-08-07 06:42:33 on dist-test-slave-gkw7"
server_key: "72e9fdfa9611df8875347d7dff3714dc"
server_key_iv: "4d4882b8e03deaeefccaa1eeac2901bb"
server_key_version: "encryptionkey@0"
I20260807 06:42:33.706674 9366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:33.720345 9366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:33.722828 9366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:33.723008 9366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:33.727192 9366 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36077
I20260807 06:42:33.727248 9457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36077 every 8 connection(s)
I20260807 06:42:33.727640 9366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:33.728572 9458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:33.731012 9458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e: Bootstrap starting.
I20260807 06:42:33.731124 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 9366
I20260807 06:42:33.731241 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:33.731482 11723 external_mini_cluster.cc:1428] Setting key 58c3d7d0bc3bf5a25f1e5757d51d3ef6
I20260807 06:42:33.731765 9458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:33.732134 9458 log.cc:824] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:33.732865 9458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e: No bootstrap required, opened a new log
I20260807 06:42:33.735147 9458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae969c920554283bd0920ab9ea1bd4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36077 } }
I20260807 06:42:33.735342 9458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:33.735376 9458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae969c920554283bd0920ab9ea1bd4e, State: Initialized, Role: FOLLOWER
I20260807 06:42:33.735529 9458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae969c920554283bd0920ab9ea1bd4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36077 } }
I20260807 06:42:33.735610 9458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:33.735632 9458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:33.735704 9458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:33.736358 9458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae969c920554283bd0920ab9ea1bd4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36077 } }
I20260807 06:42:33.736503 9458 leader_election.cc:302] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ae969c920554283bd0920ab9ea1bd4e; no voters:
I20260807 06:42:33.736696 9458 leader_election.cc:288] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:33.736737 9465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:33.736892 9465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [term 1 LEADER]: Becoming Leader. State: Replica: 2ae969c920554283bd0920ab9ea1bd4e, State: Running, Role: LEADER
I20260807 06:42:33.736982 9458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:33.736982 9465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae969c920554283bd0920ab9ea1bd4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36077 } }
I20260807 06:42:33.737372 9465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ae969c920554283bd0920ab9ea1bd4e. Latest consensus state: current_term: 1 leader_uuid: "2ae969c920554283bd0920ab9ea1bd4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae969c920554283bd0920ab9ea1bd4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36077 } } }
I20260807 06:42:33.737453 9465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:33.737707 9476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:33.737721 9469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ae969c920554283bd0920ab9ea1bd4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae969c920554283bd0920ab9ea1bd4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36077 } } }
I20260807 06:42:33.737795 9469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:33.737980 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:33.732019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:50633 (local address 127.0.0.1:36077)
0807 06:42:33.732245 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:33.732255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:33.732299 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:42:33.732846 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:33.732854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:33.732907 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:33.733064 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:33.733322 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:33.734276 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:33.735480 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:33.736000 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:33.736082 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:33.736894 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:33.736897 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:33.736898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:33.737180 (+ 282us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:33.737225 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:33.737242 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:33.737331 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:33.737451 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 06:42:33.737480 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":93,"threads_started":1}
I20260807 06:42:33.738662 9476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:33.739300 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:33.741102 9476 catalog_manager.cc:1347] Generated new cluster ID: 16670eb46bd9462e90ceb96287bf15ac
I20260807 06:42:33.741163 9476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:33.744622 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:33.740883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:34471 (local address 127.0.0.1:36077)
0807 06:42:33.740922 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:33.740924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:33.741091 (+ 167us) server_negotiation.cc:407] Connection header received
0807 06:42:33.741266 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:33.741267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:33.741280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:33.741309 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:33.741501 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:33.742267 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:33.743300 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:33.743616 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:33.743690 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:33.744363 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:33.744365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:33.744365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:33.744370 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:33.744384 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:33.744389 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:33.744457 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:33.744525 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 06:42:33.744536 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:33.782032 9476 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:33.782799 9476 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:33.791805 9476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ae969c920554283bd0920ab9ea1bd4e: Generated new TSK 0
I20260807 06:42:33.791990 9476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:33.801673 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 9366
W20260807 06:42:33.811460 9344 connection.cc:535] client connection to 127.0.0.1:36077 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36077): Connection reset by peer (error 104)
2026-08-07T06:42:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:33 dist-test-slave-gkw7 krb5kdc[9506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:33 dist-test-slave-gkw7 krb5kdc[9506](info): set up 2 sockets
Aug 07 06:42:33 dist-test-slave-gkw7 krb5kdc[9506](info): commencing operation
krb5kdc: starting...
W20260807 06:42:35.014371 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:35 dist-test-slave-gkw7 krb5kdc[9506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:42:35.138386 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:35.228667 9524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:35.228914 9524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:35.228938 9524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:35.230170 9524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:35.230224 9524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:35.230242 9524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:35.230257 9524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:35.230271 9524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:35.231839 9524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40881
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:35.232128 9524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:35.232388 9524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:35.237818 9531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:35.237824 9530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:35.237987 9524 server_base.cc:1034] running on GCE node
W20260807 06:42:35.237833 9534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:35.238354 9524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:35.238706 9524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:35.239879 9524 hybrid_clock.cc:648] HybridClock initialized: now 1786084955239866 us; error 28 us; skew 500 ppm
Aug 07 06:42:35 dist-test-slave-gkw7 krb5kdc[9506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084955, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:35.242684 9524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:42:35.243176 9524 webserver.cc:460] Webserver started at http://127.0.0.1:41013/ using document root <none> and password file <none>
I20260807 06:42:35.243403 9524 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:35.243448 9524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:35.243610 9524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:35.244694 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "e601c6aff7b84552beda4e6dc79a23a6"
format_stamp: "Formatted at 2026-08-07 06:42:35 on dist-test-slave-gkw7"
server_key: "bf4cb107a0cf41f772c9440677d67df9"
server_key_iv: "42cec1c67404f63a336caf821cdc334e"
server_key_version: "encryptionkey@0"
I20260807 06:42:35.245057 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "e601c6aff7b84552beda4e6dc79a23a6"
format_stamp: "Formatted at 2026-08-07 06:42:35 on dist-test-slave-gkw7"
server_key: "bf4cb107a0cf41f772c9440677d67df9"
server_key_iv: "42cec1c67404f63a336caf821cdc334e"
server_key_version: "encryptionkey@0"
I20260807 06:42:35.246474 9524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:35.247226 9542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:35.247426 9524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:42:35.247522 9524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "e601c6aff7b84552beda4e6dc79a23a6"
format_stamp: "Formatted at 2026-08-07 06:42:35 on dist-test-slave-gkw7"
server_key: "bf4cb107a0cf41f772c9440677d67df9"
server_key_iv: "42cec1c67404f63a336caf821cdc334e"
server_key_version: "encryptionkey@0"
I20260807 06:42:35.247605 9524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:35.250969 9524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:35.253322 9524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:35.253479 9524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:35.257480 9524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40881
I20260807 06:42:35.257539 9617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40881 every 8 connection(s)
I20260807 06:42:35.257892 9524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:35.258745 9618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:35.260979 9618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6: Bootstrap starting.
I20260807 06:42:35.261646 9618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:35.261955 9618 log.cc:824] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:35.262622 9618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6: No bootstrap required, opened a new log
I20260807 06:42:35.263670 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 9524
I20260807 06:42:35.263801 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:35.264014 11723 external_mini_cluster.cc:1428] Setting key 95669b2d8ae56bdd58e36e2c5dfc57d3
I20260807 06:42:35.264515 9618 raft_consensus.cc:348] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e601c6aff7b84552beda4e6dc79a23a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260807 06:42:35.264703 9618 raft_consensus.cc:374] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:35.264779 9618 raft_consensus.cc:729] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e601c6aff7b84552beda4e6dc79a23a6, State: Initialized, Role: FOLLOWER
I20260807 06:42:35.264916 9618 consensus_queue.cc:260] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e601c6aff7b84552beda4e6dc79a23a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260807 06:42:35.264995 9618 raft_consensus.cc:388] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:35.265017 9618 raft_consensus.cc:482] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:35.265082 9618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:35.266031 9618 raft_consensus.cc:504] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e601c6aff7b84552beda4e6dc79a23a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260807 06:42:35.266155 9618 leader_election.cc:302] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e601c6aff7b84552beda4e6dc79a23a6; no voters:
I20260807 06:42:35.266341 9618 leader_election.cc:288] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:35.266477 9624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:35.266564 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:35.264528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:42387 (local address 127.0.0.1:40881)
0807 06:42:35.264740 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:35.264749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:35.264803 (+ 54us) server_negotiation.cc:407] Connection header received
0807 06:42:35.264916 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:35.264921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:35.264945 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:35.265094 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:35.266037 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:35.266040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:35.266041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:35.266045 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:35.266102 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:35.266113 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:35.266195 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:35.266297 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 06:42:35.266369 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260807 06:42:35.266611 9618 sys_catalog.cc:564] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:35.266683 9624 raft_consensus.cc:686] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [term 1 LEADER]: Becoming Leader. State: Replica: e601c6aff7b84552beda4e6dc79a23a6, State: Running, Role: LEADER
I20260807 06:42:35.266790 9624 consensus_queue.cc:237] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e601c6aff7b84552beda4e6dc79a23a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260807 06:42:35.267277 9625 sys_catalog.cc:455] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e601c6aff7b84552beda4e6dc79a23a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e601c6aff7b84552beda4e6dc79a23a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } } }
I20260807 06:42:35.267376 9625 sys_catalog.cc:458] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:35.267513 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:35.267525 9626 sys_catalog.cc:455] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e601c6aff7b84552beda4e6dc79a23a6. Latest consensus state: current_term: 1 leader_uuid: "e601c6aff7b84552beda4e6dc79a23a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e601c6aff7b84552beda4e6dc79a23a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } } }
I20260807 06:42:35.267604 9626 sys_catalog.cc:458] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:35.267601 9637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:35.268292 9637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:35.269910 9637 catalog_manager.cc:1347] Generated new cluster ID: 8fe4151264cc40e1a006f86713b2326c
I20260807 06:42:35.269963 9637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:35.270362 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:35.269109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:48405 (local address 127.0.0.1:40881)
0807 06:42:35.269148 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:35.269149 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:35.269300 (+ 151us) server_negotiation.cc:407] Connection header received
0807 06:42:35.269388 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:35.269389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:35.269394 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:35.269421 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:35.270162 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:35.270163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:35.270163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:35.270168 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:35.270182 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:35.270186 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:35.270250 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:35.270296 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 06:42:35.270307 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:42:35.291462 9637 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:35.291680 9637 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:35.296509 9637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e601c6aff7b84552beda4e6dc79a23a6: Generated new TSK 0
I20260807 06:42:35.296747 9637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:35.306382 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 9524
2026-08-07T06:42:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:35 dist-test-slave-gkw7 krb5kdc[9662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:35 dist-test-slave-gkw7 krb5kdc[9662](info): set up 2 sockets
Aug 07 06:42:35 dist-test-slave-gkw7 krb5kdc[9662](info): commencing operation
krb5kdc: starting...
W20260807 06:42:36.521735 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:36 dist-test-slave-gkw7 krb5kdc[9662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:36.642278 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:42433
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:42433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:36.725175 9676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:36.725378 9676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:36.725401 9676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:36.726677 9676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:36.726720 9676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:36.726735 9676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:36.726748 9676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:36.726760 9676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:36.728420 9676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:42433
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:42433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:36.728682 9676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:36.728922 9676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:36.733954 9685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:36.733954 9682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:36.733995 9683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:36.734169 9676 server_base.cc:1034] running on GCE node
I20260807 06:42:36.734382 9676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:36.734707 9676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:36.735872 9676 hybrid_clock.cc:648] HybridClock initialized: now 1786084956735858 us; error 27 us; skew 500 ppm
Aug 07 06:42:36 dist-test-slave-gkw7 krb5kdc[9662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084956, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:36.742900 9676 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:36.743515 9676 webserver.cc:460] Webserver started at http://127.11.114.254:33077/ using document root <none> and password file <none>
I20260807 06:42:36.743777 9676 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:36.743824 9676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:36.743981 9676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:36.745036 9676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2"
format_stamp: "Formatted at 2026-08-07 06:42:36 on dist-test-slave-gkw7"
server_key: "26aad886a34d7f14a17a9a4501c36c9b"
server_key_iv: "0be0bda7651f6d44e0cf7fa21b178a09"
server_key_version: "encryptionkey@0"
I20260807 06:42:36.745409 9676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2"
format_stamp: "Formatted at 2026-08-07 06:42:36 on dist-test-slave-gkw7"
server_key: "26aad886a34d7f14a17a9a4501c36c9b"
server_key_iv: "0be0bda7651f6d44e0cf7fa21b178a09"
server_key_version: "encryptionkey@0"
I20260807 06:42:36.746836 9676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:36.747640 9695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:36.747823 9676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:42:36.747915 9676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2"
format_stamp: "Formatted at 2026-08-07 06:42:36 on dist-test-slave-gkw7"
server_key: "26aad886a34d7f14a17a9a4501c36c9b"
server_key_iv: "0be0bda7651f6d44e0cf7fa21b178a09"
server_key_version: "encryptionkey@0"
I20260807 06:42:36.747970 9676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:36.763123 9676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:36.765151 9676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:36.765261 9676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:36.769641 9676 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:42433
I20260807 06:42:36.769711 9778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:42433 every 8 connection(s)
I20260807 06:42:36.770087 9676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:36.771086 9779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:36.773603 9779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2: Bootstrap starting.
I20260807 06:42:36.774240 9779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:36.774562 9779 log.cc:824] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:36.775324 9779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2: No bootstrap required, opened a new log
I20260807 06:42:36.777027 9779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42433 } }
I20260807 06:42:36.777202 9779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:36.777323 9779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40f4174f34ad4bdbb4b38aa3abfe5fc2, State: Initialized, Role: FOLLOWER
I20260807 06:42:36.777501 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 9676
I20260807 06:42:36.777524 9779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42433 } }
I20260807 06:42:36.777666 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:36.777661 9779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:36.777707 9779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:36.777770 9779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:36.777890 11723 external_mini_cluster.cc:1428] Setting key 0c80f2ac8967553e8b50b06f2be946b1
I20260807 06:42:36.778975 9779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42433 } }
I20260807 06:42:36.779112 9779 leader_election.cc:302] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40f4174f34ad4bdbb4b38aa3abfe5fc2; no voters:
I20260807 06:42:36.779314 9779 leader_election.cc:288] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:36.779335 9784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:36.779487 9784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [term 1 LEADER]: Becoming Leader. State: Replica: 40f4174f34ad4bdbb4b38aa3abfe5fc2, State: Running, Role: LEADER
I20260807 06:42:36.779618 9784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42433 } }
I20260807 06:42:36.779624 9779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:36.780025 9785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42433 } } }
I20260807 06:42:36.780121 9785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:36.780280 9786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40f4174f34ad4bdbb4b38aa3abfe5fc2. Latest consensus state: current_term: 1 leader_uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4174f34ad4bdbb4b38aa3abfe5fc2" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 42433 } } }
I20260807 06:42:36.780805 9786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:36.781607 9788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:36.782359 9788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:36.784212 9788 catalog_manager.cc:1347] Generated new cluster ID: 384b14690f8542ae8b299be7571ca8fd
I20260807 06:42:36.784279 9788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:42:36 dist-test-slave-gkw7 krb5kdc[9662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:36.791092 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:36.778418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53499 (local address 127.11.114.254:42433)
0807 06:42:36.778614 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:36.778621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:36.778646 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:42:36.778737 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:36.778742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:36.778789 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:36.778898 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:36.779937 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:36.781328 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:36.782458 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:36.783155 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:36.783289 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:36.786325 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:36.786328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:36.786328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:36.786340 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:36.788365 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:36.788872 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:36.788875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:36.788876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:36.789636 (+ 760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:36.789932 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:36.789933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:36.789933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:36.790283 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:36.790435 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:36.790635 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 06:42:36.790883 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":112,"threads_started":1}
I20260807 06:42:36.792447 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:36.801640 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:36.793911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47381 (local address 127.11.114.254:42433)
0807 06:42:36.793950 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:36.793952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:36.794171 (+ 219us) server_negotiation.cc:407] Connection header received
0807 06:42:36.794231 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:36.794232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:36.794241 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:36.794295 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:36.794974 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:36.795684 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:36.796828 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:36.797155 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:36.797209 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:36.798729 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:36.798730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:36.798731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:36.798738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:36.800312 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:36.800799 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:36.800800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:36.800800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:36.800853 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:36.801099 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:36.801099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:36.801100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:36.801210 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:36.801298 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:36.801476 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 06:42:36.801557 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:42:36.822376 9788 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:36.822933 9788 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:36.827229 9788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40f4174f34ad4bdbb4b38aa3abfe5fc2: Generated new TSK 0
I20260807 06:42:36.827420 9788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:36.832705 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 9676
W20260807 06:42:36.845919 9660 connection.cc:535] client connection to 127.11.114.254:42433 recv error: Network error: failed to read from TLS socket (remote: 127.11.114.254:42433): Connection reset by peer (error 104)
2026-08-07T06:42:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:36 dist-test-slave-gkw7 krb5kdc[9816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:36 dist-test-slave-gkw7 krb5kdc[9816](info): set up 2 sockets
Aug 07 06:42:36 dist-test-slave-gkw7 krb5kdc[9816](info): commencing operation
krb5kdc: starting...
W20260807 06:42:38.076699 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:38 dist-test-slave-gkw7 krb5kdc[9816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:38.204579 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33485
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:33485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:38.288013 9830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:38.288262 9830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:38.288304 9830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:38.289440 9830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:38.289507 9830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:38.289535 9830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:38.289575 9830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:38.289635 9830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:38.291139 9830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:33485
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:38.291424 9830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:38.291659 9830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:42:38.296561 9830 server_base.cc:1034] running on GCE node
W20260807 06:42:38.296545 9843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:38.296527 9840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:38.296537 9837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:38.297024 9830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:38.297365 9830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:38.298529 9830 hybrid_clock.cc:648] HybridClock initialized: now 1786084958298521 us; error 46 us; skew 500 ppm
Aug 07 06:42:38 dist-test-slave-gkw7 krb5kdc[9816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084958, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:38.301156 9830 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:38.301654 9830 webserver.cc:460] Webserver started at http://127.11.114.254:33311/ using document root <none> and password file <none>
I20260807 06:42:38.301913 9830 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:38.301983 9830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:38.302129 9830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:38.303165 9830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "564ca3c072374e299f20b26973bc7f17"
format_stamp: "Formatted at 2026-08-07 06:42:38 on dist-test-slave-gkw7"
server_key: "c6709c7999e4457160d1d51b809c1e39"
server_key_iv: "fff3119ea46fe5597817538b5135b049"
server_key_version: "encryptionkey@0"
I20260807 06:42:38.303563 9830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "564ca3c072374e299f20b26973bc7f17"
format_stamp: "Formatted at 2026-08-07 06:42:38 on dist-test-slave-gkw7"
server_key: "c6709c7999e4457160d1d51b809c1e39"
server_key_iv: "fff3119ea46fe5597817538b5135b049"
server_key_version: "encryptionkey@0"
I20260807 06:42:38.304924 9830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:38.305748 9851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:38.305936 9830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:38.306042 9830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "564ca3c072374e299f20b26973bc7f17"
format_stamp: "Formatted at 2026-08-07 06:42:38 on dist-test-slave-gkw7"
server_key: "c6709c7999e4457160d1d51b809c1e39"
server_key_iv: "fff3119ea46fe5597817538b5135b049"
server_key_version: "encryptionkey@0"
I20260807 06:42:38.306128 9830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:38.314561 9830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:38.316576 9830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:38.316735 9830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:38.320640 9830 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:33485
I20260807 06:42:38.320709 9922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:33485 every 8 connection(s)
I20260807 06:42:38.321122 9830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:38.322060 9923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:38.324328 9923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17: Bootstrap starting.
I20260807 06:42:38.324914 9923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:38.325201 9923 log.cc:824] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:38.325951 9923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17: No bootstrap required, opened a new log
I20260807 06:42:38.327521 9923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ca3c072374e299f20b26973bc7f17" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33485 } }
I20260807 06:42:38.327670 9923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:38.327729 9923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564ca3c072374e299f20b26973bc7f17, State: Initialized, Role: FOLLOWER
I20260807 06:42:38.327899 9923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ca3c072374e299f20b26973bc7f17" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33485 } }
I20260807 06:42:38.327962 9923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:38.327983 9923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:38.328049 9923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:38.328951 9923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ca3c072374e299f20b26973bc7f17" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33485 } }
I20260807 06:42:38.329071 9923 leader_election.cc:302] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 564ca3c072374e299f20b26973bc7f17; no voters:
I20260807 06:42:38.329296 9923 leader_election.cc:288] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:38.329401 9928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:38.329598 9928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [term 1 LEADER]: Becoming Leader. State: Replica: 564ca3c072374e299f20b26973bc7f17, State: Running, Role: LEADER
I20260807 06:42:38.329618 9923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:38.329694 9928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ca3c072374e299f20b26973bc7f17" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33485 } }
I20260807 06:42:38.329797 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 9830
I20260807 06:42:38.329950 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:38.330029 9929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "564ca3c072374e299f20b26973bc7f17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ca3c072374e299f20b26973bc7f17" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33485 } } }
I20260807 06:42:38.330017 9930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 564ca3c072374e299f20b26973bc7f17. Latest consensus state: current_term: 1 leader_uuid: "564ca3c072374e299f20b26973bc7f17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ca3c072374e299f20b26973bc7f17" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33485 } } }
I20260807 06:42:38.330122 9929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:38.330127 9930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:38.330193 11723 external_mini_cluster.cc:1428] Setting key ec5ab653b3ce6f5b4afbff31aab63413
I20260807 06:42:38.330366 9939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:38.331624 9939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:38.334580 9939 catalog_manager.cc:1347] Generated new cluster ID: 3cc3ccc8befe4ef79620e7796d99df59
I20260807 06:42:38.334654 9939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:38.336792 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:38.331317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54457 (local address 127.11.114.254:33485)
0807 06:42:38.331488 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:38.331498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:38.331534 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:42:38.331654 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:38.331661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:38.331773 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:38.332175 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:38.332268 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:38.333615 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:38.334548 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:38.335191 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:38.335312 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:38.336083 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:38.336094 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:38.336095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:38.336311 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:38.336353 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:38.336368 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:38.336463 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:38.336549 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 06:42:38.336579 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":86,"threads_started":1}
I20260807 06:42:38.338003 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:38.343417 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:38.339582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34577 (local address 127.11.114.254:33485)
0807 06:42:38.339624 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:38.339627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:38.339770 (+ 143us) server_negotiation.cc:407] Connection header received
0807 06:42:38.339877 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:38.339877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:38.339888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:38.339917 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:38.340134 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:38.340852 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:38.341959 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:38.342277 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:38.342322 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:38.343143 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:38.343145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:38.343145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:38.343151 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:38.343165 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:38.343170 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:38.343201 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:38.343333 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 06:42:38.343346 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:42:38.345885 9939 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:38.346549 9939 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:38.349463 9939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 564ca3c072374e299f20b26973bc7f17: Generated new TSK 0
I20260807 06:42:38.349645 9939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:38.354856 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 9830
W20260807 06:42:38.366389 9814 connection.cc:535] client connection to 127.11.114.254:33485 recv error: Network error: failed to read from TLS socket (remote: 127.11.114.254:33485): Connection reset by peer (error 104)
2026-08-07T06:42:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:38 dist-test-slave-gkw7 krb5kdc[9972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:38 dist-test-slave-gkw7 krb5kdc[9972](info): set up 2 sockets
Aug 07 06:42:38 dist-test-slave-gkw7 krb5kdc[9972](info): commencing operation
krb5kdc: starting...
W20260807 06:42:39.575677 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:39 dist-test-slave-gkw7 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:39.701171 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:44441
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:37567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:44441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:39.783874 9989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:39.784123 9989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:39.784162 9989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:39.785326 9989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:39.785403 9989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:39.785425 9989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:39.785449 9989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:39.785476 9989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:39.786998 9989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:37567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:44441
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:44441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:39.787292 9989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:39.787530 9989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:39.792450 9999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:39.792490 9997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:39.792476 9995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:39.792724 9989 server_base.cc:1034] running on GCE node
I20260807 06:42:39.792984 9989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:39.793330 9989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:39.794487 9989 hybrid_clock.cc:648] HybridClock initialized: now 1786084959794463 us; error 40 us; skew 500 ppm
Aug 07 06:42:39 dist-test-slave-gkw7 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084959, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:39.797144 9989 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:39.797623 9989 webserver.cc:460] Webserver started at http://127.11.114.254:42229/ using document root <none> and password file <none>
I20260807 06:42:39.797873 9989 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:39.797942 9989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:39.798100 9989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:39.799130 9989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd"
format_stamp: "Formatted at 2026-08-07 06:42:39 on dist-test-slave-gkw7"
server_key: "3e7a8af56e8474433c00ae2e08c69e41"
server_key_iv: "9fe372612e2f1bf580aca8e5ea719d06"
server_key_version: "encryptionkey@0"
I20260807 06:42:39.799490 9989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd"
format_stamp: "Formatted at 2026-08-07 06:42:39 on dist-test-slave-gkw7"
server_key: "3e7a8af56e8474433c00ae2e08c69e41"
server_key_iv: "9fe372612e2f1bf580aca8e5ea719d06"
server_key_version: "encryptionkey@0"
I20260807 06:42:39.800817 9989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:39.801604 10006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:39.801801 9989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:39.801905 9989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd"
format_stamp: "Formatted at 2026-08-07 06:42:39 on dist-test-slave-gkw7"
server_key: "3e7a8af56e8474433c00ae2e08c69e41"
server_key_iv: "9fe372612e2f1bf580aca8e5ea719d06"
server_key_version: "encryptionkey@0"
I20260807 06:42:39.801987 9989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:39.805197 9989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:39.807637 9989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:39.807803 9989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:39.811676 9989 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:44441
I20260807 06:42:39.811705 10082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:44441 every 8 connection(s)
I20260807 06:42:39.812053 9989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:39.813038 10083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:39.815323 10083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd: Bootstrap starting.
I20260807 06:42:39.815851 10083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:39.816079 10083 log.cc:824] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:39.816543 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 9989
I20260807 06:42:39.816718 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:39.816839 10083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd: No bootstrap required, opened a new log
I20260807 06:42:39.816972 11723 external_mini_cluster.cc:1428] Setting key 1450a0df44ae5e69162a840422ecb46b
I20260807 06:42:39.819391 10083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44441 } }
I20260807 06:42:39.819591 10083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:39.819650 10083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0d7145a9574dc6a7dd742e8ddfd7fd, State: Initialized, Role: FOLLOWER
I20260807 06:42:39.819779 10083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44441 } }
I20260807 06:42:39.819890 10083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:39.819936 10083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:39.819945 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:39.817612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53385 (local address 127.11.114.254:44441)
0807 06:42:39.817807 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:39.817815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:39.817842 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:42:39.818367 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:39.818375 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:39.818401 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:39.818526 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:39.819214 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:39.819218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:39.819218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:39.819222 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:39.819260 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:39.819271 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:39.819330 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:39.819424 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 06:42:39.819480 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260807 06:42:39.819983 10083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:39.820701 10083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44441 } }
I20260807 06:42:39.820813 10083 leader_election.cc:302] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b0d7145a9574dc6a7dd742e8ddfd7fd; no voters:
I20260807 06:42:39.820991 10083 leader_election.cc:288] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:39.821009 10088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:39.821010 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:39.821122 10088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [term 1 LEADER]: Becoming Leader. State: Replica: 0b0d7145a9574dc6a7dd742e8ddfd7fd, State: Running, Role: LEADER
I20260807 06:42:39.821270 10083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:39.821257 10088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44441 } }
I20260807 06:42:39.821921 10090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44441 } } }
I20260807 06:42:39.822029 10090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:39.822417 10089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b0d7145a9574dc6a7dd742e8ddfd7fd. Latest consensus state: current_term: 1 leader_uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d7145a9574dc6a7dd742e8ddfd7fd" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 44441 } } }
I20260807 06:42:39.822499 10089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:39.822842 10103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:39.823343 10103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:39.824901 10103 catalog_manager.cc:1347] Generated new cluster ID: fe4652f31d36486696dc006e6e9bf85e
I20260807 06:42:39.824978 10103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:39.826784 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:39.824215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33287 (local address 127.11.114.254:44441)
0807 06:42:39.824397 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:39.824399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:39.825727 (+ 1328us) server_negotiation.cc:407] Connection header received
0807 06:42:39.825791 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:39.825792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:39.825798 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:39.825861 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:39.826543 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:39.826544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:39.826545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:39.826549 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:39.826565 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:39.826570 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:39.826626 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:39.826704 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 06:42:39.826717 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:42:39.843613 10103 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:39.843852 10103 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:39.853624 10103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b0d7145a9574dc6a7dd742e8ddfd7fd: Generated new TSK 0
I20260807 06:42:39.853842 10103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:39.863272 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 9989
2026-08-07T06:42:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:39 dist-test-slave-gkw7 krb5kdc[10128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:39 dist-test-slave-gkw7 krb5kdc[10128](info): set up 2 sockets
Aug 07 06:42:39 dist-test-slave-gkw7 krb5kdc[10128](info): commencing operation
krb5kdc: starting...
W20260807 06:42:41.084314 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:41 dist-test-slave-gkw7 krb5kdc[10128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:41.216663 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39117
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:45605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:39117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:41.303936 10145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:41.304147 10145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:41.304169 10145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:41.305320 10145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:41.305363 10145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:41.305378 10145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:41.305392 10145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:41.305403 10145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:41.306990 10145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:45605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:39117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:39117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:41.307257 10145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:41.307492 10145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:41.312386 10151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:41.312407 10156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:41.312529 10145 server_base.cc:1034] running on GCE node
W20260807 06:42:41.312433 10153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:41.312948 10145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:41.313215 10145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:41.314389 10145 hybrid_clock.cc:648] HybridClock initialized: now 1786084961314381 us; error 37 us; skew 500 ppm
Aug 07 06:42:41 dist-test-slave-gkw7 krb5kdc[10128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084961, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:41.316977 10145 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:41.317416 10145 webserver.cc:460] Webserver started at http://127.11.114.254:39379/ using document root <none> and password file <none>
I20260807 06:42:41.317720 10145 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:41.317793 10145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:41.317955 10145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:41.319043 10145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "c6160722582646f09d229b3227744b09"
format_stamp: "Formatted at 2026-08-07 06:42:41 on dist-test-slave-gkw7"
server_key: "3f55bcb2803188e815880ae8f8b23c87"
server_key_iv: "ae058359b189c8e37de8e3f6c1aa036e"
server_key_version: "encryptionkey@0"
I20260807 06:42:41.319422 10145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "c6160722582646f09d229b3227744b09"
format_stamp: "Formatted at 2026-08-07 06:42:41 on dist-test-slave-gkw7"
server_key: "3f55bcb2803188e815880ae8f8b23c87"
server_key_iv: "ae058359b189c8e37de8e3f6c1aa036e"
server_key_version: "encryptionkey@0"
I20260807 06:42:41.320767 10145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:41.321504 10166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:41.321720 10145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:41.321835 10145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "c6160722582646f09d229b3227744b09"
format_stamp: "Formatted at 2026-08-07 06:42:41 on dist-test-slave-gkw7"
server_key: "3f55bcb2803188e815880ae8f8b23c87"
server_key_iv: "ae058359b189c8e37de8e3f6c1aa036e"
server_key_version: "encryptionkey@0"
I20260807 06:42:41.321946 10145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:41.333794 10145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:41.336254 10145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:41.336431 10145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:41.340597 10145 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:39117
I20260807 06:42:41.340634 10237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:39117 every 8 connection(s)
I20260807 06:42:41.341012 10145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:41.342056 10238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:41.342433 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 10145
I20260807 06:42:41.342562 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:41.342765 11723 external_mini_cluster.cc:1428] Setting key 157f9698aa1ba2c23fa220c2d29816ad
I20260807 06:42:41.345273 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09: Bootstrap starting.
I20260807 06:42:41.345934 10238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:41.346223 10238 log.cc:824] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:41.346928 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09: No bootstrap required, opened a new log
I20260807 06:42:41.348729 10238 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6160722582646f09d229b3227744b09" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39117 } }
I20260807 06:42:41.348896 10238 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:41.348935 10238 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6160722582646f09d229b3227744b09, State: Initialized, Role: FOLLOWER
I20260807 06:42:41.349063 10238 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6160722582646f09d229b3227744b09" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39117 } }
I20260807 06:42:41.349134 10238 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:41.349162 10238 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:41.349201 10238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 06:42:41 dist-test-slave-gkw7 krb5kdc[10128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:41.349937 10238 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6160722582646f09d229b3227744b09" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39117 } }
I20260807 06:42:41.350060 10238 leader_election.cc:302] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6160722582646f09d229b3227744b09; no voters:
I20260807 06:42:41.350278 10238 leader_election.cc:288] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:41.350317 10246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:41.350512 10246 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [term 1 LEADER]: Becoming Leader. State: Replica: c6160722582646f09d229b3227744b09, State: Running, Role: LEADER
I20260807 06:42:41.350564 10238 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:41.350708 10246 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6160722582646f09d229b3227744b09" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39117 } }
I20260807 06:42:41.351143 10246 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6160722582646f09d229b3227744b09. Latest consensus state: current_term: 1 leader_uuid: "c6160722582646f09d229b3227744b09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6160722582646f09d229b3227744b09" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39117 } } }
I20260807 06:42:41.351380 10246 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:41.351308 10247 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6160722582646f09d229b3227744b09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6160722582646f09d229b3227744b09" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 39117 } } }
I20260807 06:42:41.351816 10247 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:41.352013 10257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:41.352759 10257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:41.354389 10257 catalog_manager.cc:1347] Generated new cluster ID: e7c03aeabe7145f094f3eebb4ffddd68
I20260807 06:42:41.354444 10257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:41.354583 10244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:41.343307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52629 (local address 127.11.114.254:39117)
0807 06:42:41.343498 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:41.343507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:41.343533 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:42:41.343944 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:41.343950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:41.343996 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:41.344114 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:41.345015 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:41.346045 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:41.346904 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:41.347366 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:41.347487 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:41.350193 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:41.350198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:41.350198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:41.350212 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:41.352516 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:41.352957 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:41.352960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:41.352961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:41.353063 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:41.353298 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:41.353299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:41.353299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:41.353660 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:41.353818 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:41.354208 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 06:42:41.354350 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260807 06:42:41.355616 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:41.368884 10244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:41.361610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59395 (local address 127.11.114.254:39117)
0807 06:42:41.361671 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:41.361673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:41.361682 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:41.361704 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:41.361705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:41.361714 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:41.361769 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:41.362464 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:41.363266 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:41.364244 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:41.364584 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:41.364656 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:41.366062 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:41.366063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:41.366063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:41.366071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:41.367696 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:41.368073 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:41.368074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:41.368075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:41.368136 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:41.368331 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:41.368332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:41.368332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:41.368476 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:41.368574 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:41.368694 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 06:42:41.368785 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:42:41.371622 10257 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:41.372146 10257 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:41.381768 10257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6160722582646f09d229b3227744b09: Generated new TSK 0
I20260807 06:42:41.382011 10257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:41.384591 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 10145
2026-08-07T06:42:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:41 dist-test-slave-gkw7 krb5kdc[10284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:41 dist-test-slave-gkw7 krb5kdc[10284](info): set up 2 sockets
Aug 07 06:42:41 dist-test-slave-gkw7 krb5kdc[10284](info): commencing operation
krb5kdc: starting...
W20260807 06:42:42.629729 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:42 dist-test-slave-gkw7 krb5kdc[10284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:42.761072 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:41553
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:41553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:42.844340 10301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:42.844601 10301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:42.844640 10301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:42.845839 10301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:42.845917 10301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:42.845942 10301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:42.845964 10301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:42.845993 10301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:42.848067 10301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:41553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:41553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:42.848361 10301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:42.848611 10301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:42.853566 10307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:42.853567 10311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:42.853641 10308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:42.854103 10301 server_base.cc:1034] running on GCE node
I20260807 06:42:42.854308 10301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:42.854547 10301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:42.855722 10301 hybrid_clock.cc:648] HybridClock initialized: now 1786084962855712 us; error 38 us; skew 500 ppm
Aug 07 06:42:42 dist-test-slave-gkw7 krb5kdc[10284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084962, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:42.858425 10301 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:42.858904 10301 webserver.cc:460] Webserver started at http://127.11.114.254:45681/ using document root <none> and password file <none>
I20260807 06:42:42.859091 10301 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:42.859140 10301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:42.859230 10301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:42.860255 10301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "b8ed9e5080ae429195821d69ad96f2e4"
format_stamp: "Formatted at 2026-08-07 06:42:42 on dist-test-slave-gkw7"
server_key: "e30ac1662850e0e7798094bf08f578b7"
server_key_iv: "374879be1600b0862080894f49f85c93"
server_key_version: "encryptionkey@0"
I20260807 06:42:42.860569 10301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "b8ed9e5080ae429195821d69ad96f2e4"
format_stamp: "Formatted at 2026-08-07 06:42:42 on dist-test-slave-gkw7"
server_key: "e30ac1662850e0e7798094bf08f578b7"
server_key_iv: "374879be1600b0862080894f49f85c93"
server_key_version: "encryptionkey@0"
I20260807 06:42:42.861954 10301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:42.862762 10321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:42.863011 10301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:42.863121 10301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "b8ed9e5080ae429195821d69ad96f2e4"
format_stamp: "Formatted at 2026-08-07 06:42:42 on dist-test-slave-gkw7"
server_key: "e30ac1662850e0e7798094bf08f578b7"
server_key_iv: "374879be1600b0862080894f49f85c93"
server_key_version: "encryptionkey@0"
I20260807 06:42:42.863178 10301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:42.902175 10301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:42.904706 10301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:42.904832 10301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:42.909000 10301 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:41553
I20260807 06:42:42.909058 10391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:41553 every 8 connection(s)
I20260807 06:42:42.909435 10301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:42.910517 10394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:42.913141 10394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4: Bootstrap starting.
I20260807 06:42:42.913872 10394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:42.914203 10394 log.cc:824] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:42.915046 10394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4: No bootstrap required, opened a new log
I20260807 06:42:42.916963 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 10301
I20260807 06:42:42.916932 10394 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ed9e5080ae429195821d69ad96f2e4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41553 } }
I20260807 06:42:42.917107 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:42.917133 10394 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:42.917186 10394 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8ed9e5080ae429195821d69ad96f2e4, State: Initialized, Role: FOLLOWER
I20260807 06:42:42.917310 11723 external_mini_cluster.cc:1428] Setting key c920eb4c027acacd53aabe9522df529d
I20260807 06:42:42.917359 10394 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ed9e5080ae429195821d69ad96f2e4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41553 } }
I20260807 06:42:42.917459 10394 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:42.917510 10394 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:42.917609 10394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:42.918924 10394 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ed9e5080ae429195821d69ad96f2e4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41553 } }
I20260807 06:42:42.919061 10394 leader_election.cc:302] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8ed9e5080ae429195821d69ad96f2e4; no voters:
I20260807 06:42:42.919269 10394 leader_election.cc:288] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:42.919312 10400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:42.919488 10400 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [term 1 LEADER]: Becoming Leader. State: Replica: b8ed9e5080ae429195821d69ad96f2e4, State: Running, Role: LEADER
I20260807 06:42:42.919576 10394 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:42.919643 10400 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ed9e5080ae429195821d69ad96f2e4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41553 } }
I20260807 06:42:42.919968 10400 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8ed9e5080ae429195821d69ad96f2e4. Latest consensus state: current_term: 1 leader_uuid: "b8ed9e5080ae429195821d69ad96f2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ed9e5080ae429195821d69ad96f2e4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41553 } } }
I20260807 06:42:42.919979 10402 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8ed9e5080ae429195821d69ad96f2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ed9e5080ae429195821d69ad96f2e4" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 41553 } } }
I20260807 06:42:42.920104 10400 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:42.920132 10402 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:42.920351 10407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:42.921144 10407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:42.923174 10407 catalog_manager.cc:1347] Generated new cluster ID: 39012b8c017b4ee5bd887fd8218275e8
I20260807 06:42:42.923231 10407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:42.923707 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:42.917875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49277 (local address 127.11.114.254:41553)
0807 06:42:42.918053 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:42.918095 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:42.918125 (+ 30us) server_negotiation.cc:407] Connection header received
0807 06:42:42.918279 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:42.918284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:42.918328 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:42.918439 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:42.918698 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:42.920016 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:42.921176 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:42.921880 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:42.922005 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:42.922996 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:42.923000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:42.923000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:42.923224 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:42.923268 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:42.923283 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:42.923351 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:42.923443 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 06:42:42.923506 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260807 06:42:42.924642 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:42.933715 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:42.929861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39019 (local address 127.11.114.254:41553)
0807 06:42:42.929894 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:42.929896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:42.930047 (+ 151us) server_negotiation.cc:407] Connection header received
0807 06:42:42.930087 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:42.930088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:42.930103 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:42.930179 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:42.930321 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:42.931180 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:42.932177 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:42.932545 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:42.932630 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:42.933392 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:42.933393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:42.933393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:42.933399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:42.933418 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:42.933424 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:42.933491 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:42.933586 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 06:42:42.933618 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":16}
I20260807 06:42:42.942749 10407 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:42.943344 10407 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:42.954665 10407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8ed9e5080ae429195821d69ad96f2e4: Generated new TSK 0
I20260807 06:42:42.954895 10407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:42.957625 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 10301
2026-08-07T06:42:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:43 dist-test-slave-gkw7 krb5kdc[10437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:43 dist-test-slave-gkw7 krb5kdc[10437](info): set up 2 sockets
Aug 07 06:42:43 dist-test-slave-gkw7 krb5kdc[10437](info): commencing operation
krb5kdc: starting...
W20260807 06:42:44.279749 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:44 dist-test-slave-gkw7 krb5kdc[10437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:44.406201 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33707
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:44.490515 10452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:44.490731 10452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:44.490761 10452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:44.491904 10452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:44.491946 10452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:44.491961 10452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:44.491972 10452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:44.491984 10452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:44.493413 10452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:33707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:44.493673 10452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:44.493858 10452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:44.499224 10458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:44.499209 10460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:44.499209 10463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:44.499500 10452 server_base.cc:1034] running on GCE node
I20260807 06:42:44.499744 10452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:44.500036 10452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:44.501205 10452 hybrid_clock.cc:648] HybridClock initialized: now 1786084964501193 us; error 33 us; skew 500 ppm
Aug 07 06:42:44 dist-test-slave-gkw7 krb5kdc[10437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084964, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:44.503906 10452 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:44.504381 10452 webserver.cc:460] Webserver started at http://127.11.114.254:42845/ using document root <none> and password file <none>
I20260807 06:42:44.504611 10452 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:44.504679 10452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:44.504839 10452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:44.505990 10452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "d625879a504e4ad29ba4e20f95b9d832"
format_stamp: "Formatted at 2026-08-07 06:42:44 on dist-test-slave-gkw7"
server_key: "7d6b59d233e119f3226ed7d1be9d5c7d"
server_key_iv: "68a5e07ed7b64bfa82cc1a3481280d66"
server_key_version: "encryptionkey@0"
I20260807 06:42:44.506366 10452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "d625879a504e4ad29ba4e20f95b9d832"
format_stamp: "Formatted at 2026-08-07 06:42:44 on dist-test-slave-gkw7"
server_key: "7d6b59d233e119f3226ed7d1be9d5c7d"
server_key_iv: "68a5e07ed7b64bfa82cc1a3481280d66"
server_key_version: "encryptionkey@0"
I20260807 06:42:44.507762 10452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 06:42:44.508550 10474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:44.508754 10452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:44.508864 10452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "d625879a504e4ad29ba4e20f95b9d832"
format_stamp: "Formatted at 2026-08-07 06:42:44 on dist-test-slave-gkw7"
server_key: "7d6b59d233e119f3226ed7d1be9d5c7d"
server_key_iv: "68a5e07ed7b64bfa82cc1a3481280d66"
server_key_version: "encryptionkey@0"
I20260807 06:42:44.508940 10452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:44.512310 10452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:44.514678 10452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:44.514842 10452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:44.519112 10452 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:33707
I20260807 06:42:44.519145 10552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:33707 every 8 connection(s)
I20260807 06:42:44.519491 10452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:44.520455 10553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:44.521350 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 10452
I20260807 06:42:44.521478 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:44.521706 11723 external_mini_cluster.cc:1428] Setting key 574173f819cb33d90844fdfb94b77657
I20260807 06:42:44.523922 10553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832: Bootstrap starting.
I20260807 06:42:44.524451 10557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:44.522259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59015 (local address 127.11.114.254:33707)
0807 06:42:44.522466 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:44.522476 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:44.522508 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:42:44.523014 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:44.523024 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:44.523070 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:44.523225 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:44.523856 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:44.523860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:44.523860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:44.523865 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:44.523911 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:44.523924 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:44.524004 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:44.524203 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 06:42:44.524252 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260807 06:42:44.524641 10553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:44.524909 10553 log.cc:824] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:44.525600 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:44.525725 10553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832: No bootstrap required, opened a new log
I20260807 06:42:44.528366 10557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:44.527227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47415 (local address 127.11.114.254:33707)
0807 06:42:44.527293 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:44.527295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:44.527406 (+ 111us) server_negotiation.cc:407] Connection header received
0807 06:42:44.527471 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:44.527471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:44.527476 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:44.527544 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:44.528117 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:44.528118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:44.528118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:42:44.528122 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:44.528138 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:42:44.528143 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:44.528196 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:44.528247 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 06:42:44.528259 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:42:44.528409 10553 raft_consensus.cc:348] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d625879a504e4ad29ba4e20f95b9d832" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33707 } }
I20260807 06:42:44.528604 10553 raft_consensus.cc:374] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:42:44.528654 10553 raft_consensus.cc:729] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d625879a504e4ad29ba4e20f95b9d832, State: Initialized, Role: FOLLOWER
I20260807 06:42:44.528797 10553 consensus_queue.cc:260] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d625879a504e4ad29ba4e20f95b9d832" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33707 } }
I20260807 06:42:44.528909 10553 raft_consensus.cc:388] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:42:44.528956 10553 raft_consensus.cc:482] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:42:44.528998 10553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:44.529572 10553 raft_consensus.cc:504] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d625879a504e4ad29ba4e20f95b9d832" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33707 } }
I20260807 06:42:44.529693 10553 leader_election.cc:302] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d625879a504e4ad29ba4e20f95b9d832; no voters:
I20260807 06:42:44.529858 10553 leader_election.cc:288] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:42:44.529889 10568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:44.530074 10568 raft_consensus.cc:686] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [term 1 LEADER]: Becoming Leader. State: Replica: d625879a504e4ad29ba4e20f95b9d832, State: Running, Role: LEADER
I20260807 06:42:44.530105 10553 sys_catalog.cc:564] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:44.530197 10568 consensus_queue.cc:237] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d625879a504e4ad29ba4e20f95b9d832" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33707 } }
I20260807 06:42:44.530620 10569 sys_catalog.cc:455] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d625879a504e4ad29ba4e20f95b9d832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d625879a504e4ad29ba4e20f95b9d832" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33707 } } }
I20260807 06:42:44.530645 10570 sys_catalog.cc:455] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d625879a504e4ad29ba4e20f95b9d832. Latest consensus state: current_term: 1 leader_uuid: "d625879a504e4ad29ba4e20f95b9d832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d625879a504e4ad29ba4e20f95b9d832" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 33707 } } }
I20260807 06:42:44.530814 10570 sys_catalog.cc:458] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:44.531014 10569 sys_catalog.cc:458] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832 [sys.catalog]: This master's current role is: LEADER
W20260807 06:42:44.531920 10588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:42:44.531986 10588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:42:44.532083 10579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:44.532274 10579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:44.533766 10579 catalog_manager.cc:1347] Generated new cluster ID: 854cc2fd90a840c7ac321e8e31cd6be9
I20260807 06:42:44.533836 10579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:44.551622 10579 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:44.551846 10579 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:44.561967 10579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d625879a504e4ad29ba4e20f95b9d832: Generated new TSK 0
I20260807 06:42:44.562184 10579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:44.567124 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 10452
2026-08-07T06:42:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1604 ms)
[----------] 18 tests from AuthTokenIssuingTest (27758 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:44 dist-test-slave-gkw7 krb5kdc[10595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:44 dist-test-slave-gkw7 krb5kdc[10595](info): set up 2 sockets
Aug 07 06:42:44 dist-test-slave-gkw7 krb5kdc[10595](info): commencing operation
krb5kdc: starting...
W20260807 06:42:45.888190 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:45 dist-test-slave-gkw7 krb5kdc[10595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:46.016108 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:46591
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:46591,127.11.114.253:41691,127.11.114.252:34035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:46.100301 10613 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:42:46.100579 10613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:46.100611 10613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:46.100656 10613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:46.101945 10613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:46.102016 10613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:46.102042 10613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:46.102072 10613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:46.102120 10613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:46.103639 10613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:46591,127.11.114.253:41691,127.11.114.252:34035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:46591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:46.103941 10613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:46.104228 10613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:46.109207 10621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:46.109282 10624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:46.109207 10620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:46.109642 10613 server_base.cc:1034] running on GCE node
I20260807 06:42:46.109939 10613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:46.110297 10613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:46.111492 10613 hybrid_clock.cc:648] HybridClock initialized: now 1786084966111475 us; error 48 us; skew 500 ppm
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:46.114277 10613 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:46.114707 10613 webserver.cc:460] Webserver started at http://127.11.114.254:45659/ using document root <none> and password file <none>
I20260807 06:42:46.114907 10613 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:46.114951 10613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:46.115063 10613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:46.116205 10613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "eedfd7b7d5fb3e2217f0a3bdbe0c5fb6"
server_key_iv: "4390a9933805692f9f36c965492b1e90"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.116549 10613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "eedfd7b7d5fb3e2217f0a3bdbe0c5fb6"
server_key_iv: "4390a9933805692f9f36c965492b1e90"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.118098 10613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:42:46.118832 10634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:46.119033 10613 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:42:46.119127 10613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "eedfd7b7d5fb3e2217f0a3bdbe0c5fb6"
server_key_iv: "4390a9933805692f9f36c965492b1e90"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.119186 10613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:46.132524 10613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:46.134974 10613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:46.135097 10613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:46.139011 10613 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:46591
I20260807 06:42:46.139074 10705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:46591 every 8 connection(s)
I20260807 06:42:46.139448 10613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:46.140377 10706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:46.142066 10706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:46.142377 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 10613
I20260807 06:42:46.142496 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:46.142681 11723 external_mini_cluster.cc:1428] Setting key c4f5fd9dffd114083dda89979426759c
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.253@KRBTEST.COM; defaulting to no policy
I20260807 06:42:46.152318 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.142952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46591 (local address 127.0.0.1:41130)
0807 06:42:46.143124 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.143166 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.143621 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.143922 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.143924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.144169 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.144337 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.144341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.145376 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.145377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.146131 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.146134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.146650 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.146651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.146717 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.147281 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.147289 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.148689 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.150841 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.150847 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.150849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.151043 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.151332 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.151333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.151333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.151417 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.151769 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.151775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.151834 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.151911 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.152089 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260807 06:42:46.152325 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.142978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.11.114.254:46591)
0807 06:42:46.143165 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.143175 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.143255 (+ 80us) server_negotiation.cc:407] Connection header received
0807 06:42:46.143778 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.143780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.143804 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.143924 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.144442 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.145268 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.146224 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.146547 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.146679 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.148784 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.148792 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.148792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.148800 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.150754 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.151145 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.151148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.151149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.151256 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.151493 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.151494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.151494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.151714 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.151778 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.151993 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.152166 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260807 06:42:46.153151 10706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:46.153616 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.153447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49872)
0807 06:42:46.153495 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.153580 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.253@KRBTEST.COM" created.
W20260807 06:42:46.153772 10638 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
I20260807 06:42:46.154263 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.154106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49874)
0807 06:42:46.154190 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.154225 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 06:42:46.154397 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:41691: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab.
Entry for principal kudu/127.11.114.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab.
I20260807 06:42:46.168841 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.253:41691
--webserver_interface=127.11.114.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab
--principal=kudu/127.11.114.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:46591,127.11.114.253:41691,127.11.114.252:34035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:42:46.203653 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } attempt: 1
I20260807 06:42:46.204151 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.204008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49876)
0807 06:42:46.204079 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.204120 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 06:42:46.204530 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.204424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49884)
0807 06:42:46.204467 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.204512 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 06:42:46.204615 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:41691: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
W20260807 06:42:46.254721 10715 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:42:46.254999 10715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:46.255056 10715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:46.255133 10715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:46.256326 10715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:46.256403 10715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:46.256433 10715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:46.256467 10715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:46.256500 10715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:46.258085 10715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:46591,127.11.114.253:41691,127.11.114.252:34035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab
--principal=kudu/127.11.114.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.253:41691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.114.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:46.258384 10715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:46.258653 10715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:46.263870 10720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:46.263904 10723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:46.263890 10721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:46.264065 10715 server_base.cc:1034] running on GCE node
I20260807 06:42:46.264577 10715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:46.264842 10715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:46.265991 10715 hybrid_clock.cc:648] HybridClock initialized: now 1786084966265970 us; error 35 us; skew 500 ppm
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:46.268560 10715 init.cc:377] Logged in from keytab as kudu/127.11.114.253@KRBTEST.COM (short username kudu)
I20260807 06:42:46.268945 10715 webserver.cc:460] Webserver started at http://127.11.114.253:39209/ using document root <none> and password file <none>
I20260807 06:42:46.269129 10715 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:46.269169 10715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:46.269271 10715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:46.270361 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/data/instance:
uuid: "44a545848a674792ad8e844c70837201"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "4d7262bacfd65b93b4ca029b128904d1"
server_key_iv: "cc196f1c2a03805d239f0503cbcb90c6"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.270686 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/wal/instance:
uuid: "44a545848a674792ad8e844c70837201"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "4d7262bacfd65b93b4ca029b128904d1"
server_key_iv: "cc196f1c2a03805d239f0503cbcb90c6"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.271991 10715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:42:46.272724 10733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:46.272732 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } attempt: 2
I20260807 06:42:46.272984 10715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:42:46.273061 10715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/wal
uuid: "44a545848a674792ad8e844c70837201"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "4d7262bacfd65b93b4ca029b128904d1"
server_key_iv: "cc196f1c2a03805d239f0503cbcb90c6"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.273146 10715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:46.273211 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.273039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49890)
0807 06:42:46.273135 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.273178 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 06:42:46.273614 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.273484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49898)
0807 06:42:46.273559 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.273591 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 06:42:46.273718 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:41691: Network error: Client connection negotiation failed: client connection to 127.11.114.253:41691: connect: Connection refused (error 111)
I20260807 06:42:46.288604 10715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:46.290959 10715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:46.291131 10715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:46.295433 10715 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.253:41691
I20260807 06:42:46.295512 10802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.253:41691 every 8 connection(s)
I20260807 06:42:46.295874 10715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
I20260807 06:42:46.296797 10803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:46.298590 10803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:46.304369 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 10715
I20260807 06:42:46.304445 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-1/wal/instance
I20260807 06:42:46.304692 11723 external_mini_cluster.cc:1428] Setting key 67584890e5fc71b99ee028b138a32efb
I20260807 06:42:46.308529 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.299341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.11.114.254:46591)
0807 06:42:46.299407 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.299409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.299603 (+ 194us) server_negotiation.cc:407] Connection header received
0807 06:42:46.300065 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.300067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.300078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.300139 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.300740 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.301218 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.302263 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.302462 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.302512 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.304963 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.304965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.304965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.304974 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.306836 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.307416 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.307417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.307417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.307471 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.307783 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.307783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.307783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.307887 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.307936 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.308287 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.308385 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:42:46.308545 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.299293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46591 (local address 127.0.0.1:54964)
0807 06:42:46.299465 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.299508 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.299977 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.300204 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.300208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.300454 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.300657 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.300662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.301318 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.301319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.302161 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.302164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.302552 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.302554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.302663 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.303132 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.303141 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.304828 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.306939 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.306947 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.306950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.307331 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.307541 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.307542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.307542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.307708 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.307957 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.307963 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.308144 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.308241 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.308372 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260807 06:42:46.309243 10803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.114.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.252@KRBTEST.COM" created.
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:42:46.320125 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.309634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49900 (local address 127.11.114.253:41691)
0807 06:42:46.309819 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.309823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.309839 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:42:46.309857 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.309859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.309869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.309960 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.310250 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.311002 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.311798 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.312154 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.312254 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.316843 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.316853 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.316854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.316862 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.318646 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.319101 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.319105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.319105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.319169 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.319389 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.319390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.319390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.319547 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.319622 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.319799 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.319940 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
I20260807 06:42:46.320150 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.309580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49900)
0807 06:42:46.309633 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.309644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.309695 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.309960 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.309961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.310113 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.310161 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.310163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.311122 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.311123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.311688 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.311689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.312253 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.312254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.312327 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.312792 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.312797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.316751 (+ 3954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.318739 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.318740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.318740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.318986 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.319247 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.319247 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.319248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.319322 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.319621 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.319622 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.319676 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.319761 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.319883 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 06:42:46.321286 10803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:46.321789 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.321617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50894)
0807 06:42:46.321674 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.321733 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 06:42:46.321988 10734 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
I20260807 06:42:46.322482 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.322370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50896)
0807 06:42:46.322420 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.322452 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:42:46.322628 10803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:34035: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab.
Entry for principal kudu/127.11.114.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab.
I20260807 06:42:46.333902 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.252:34035
--webserver_interface=127.11.114.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab
--principal=kudu/127.11.114.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:46591,127.11.114.253:41691,127.11.114.252:34035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:42:46.364887 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } attempt: 3
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:42:46.371888 10803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } attempt: 1
I20260807 06:42:46.372330 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.372173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50908)
0807 06:42:46.372241 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.372288 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 06:42:46.372752 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.372617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50918)
0807 06:42:46.372678 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.372714 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 06:42:46.372910 10803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:34035: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
I20260807 06:42:46.373643 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.365492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49904 (local address 127.11.114.253:41691)
0807 06:42:46.365612 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.365631 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.365675 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:42:46.365783 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.365785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.365803 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.365883 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.366278 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.366833 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.367643 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.367923 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.368048 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.370510 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.370512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.370513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.370521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.372275 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.372744 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.372745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.372746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.372848 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.373089 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.373090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.373091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.373215 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.373276 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.373453 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.373516 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 06:42:46.373703 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.365373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49904)
0807 06:42:46.365469 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.365615 (+ 146us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.365700 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.365891 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.365892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.366104 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.366176 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.366178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.366927 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.366928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.367541 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.367543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.368053 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.368054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.368158 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.368794 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.368799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.370411 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.372376 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.372377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.372377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.372628 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.372942 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.372943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.372944 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.373021 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.373287 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.373288 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.373346 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.373438 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.373595 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 06:42:46.374264 10706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:46.374715 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.374566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50920)
0807 06:42:46.374651 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.374685 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 06:42:46.375120 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.375021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50936)
0807 06:42:46.375047 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.375069 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260807 06:42:46.375236 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:34035: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
I20260807 06:42:46.406418 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } attempt: 1
I20260807 06:42:46.406967 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.406816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50938)
0807 06:42:46.406901 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.406936 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 06:42:46.407426 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.407316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50942)
0807 06:42:46.407369 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.407405 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 06:42:46.407523 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:34035: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
W20260807 06:42:46.422976 10811 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:42:46.423274 10811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:46.423331 10811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:46.423381 10811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:46.424588 10811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:46.424640 10811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:46.424656 10811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:46.424671 10811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:46.424685 10811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:46.426329 10811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:46591,127.11.114.253:41691,127.11.114.252:34035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab
--principal=kudu/127.11.114.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.252:34035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.114.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:46.426623 10811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:46.426888 10811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:46.432458 10821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:46.432550 10816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:46.432504 10817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:46.432757 10811 server_base.cc:1034] running on GCE node
I20260807 06:42:46.433034 10811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:46.433346 10811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:46.434516 10811 hybrid_clock.cc:648] HybridClock initialized: now 1786084966434510 us; error 42 us; skew 500 ppm
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:46.437233 10811 init.cc:377] Logged in from keytab as kudu/127.11.114.252@KRBTEST.COM (short username kudu)
I20260807 06:42:46.437808 10811 webserver.cc:460] Webserver started at http://127.11.114.252:46723/ using document root <none> and password file <none>
I20260807 06:42:46.438078 10811 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:46.438150 10811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:46.438309 10811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:46.439420 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/data/instance:
uuid: "eec8226229a442738166068afa5b908f"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "c5aa3246304f90b6dc65803f75dab9fb"
server_key_iv: "7fd05e96310b90e23691030721233c55"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.439801 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/wal/instance:
uuid: "eec8226229a442738166068afa5b908f"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "c5aa3246304f90b6dc65803f75dab9fb"
server_key_iv: "7fd05e96310b90e23691030721233c55"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.441085 10803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } attempt: 2
I20260807 06:42:46.441291 10811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:46.441486 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.441362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50958)
0807 06:42:46.441424 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.441461 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 06:42:46.441854 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.441736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50974)
0807 06:42:46.441799 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.441824 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:42:46.441974 10803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:34035: Network error: Client connection negotiation failed: client connection to 127.11.114.252:34035: connect: Connection refused (error 111)
I20260807 06:42:46.442250 10828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:46.442441 10811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:46.442540 10811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/wal
uuid: "eec8226229a442738166068afa5b908f"
format_stamp: "Formatted at 2026-08-07 06:42:46 on dist-test-slave-gkw7"
server_key: "c5aa3246304f90b6dc65803f75dab9fb"
server_key_iv: "7fd05e96310b90e23691030721233c55"
server_key_version: "encryptionkey@0"
I20260807 06:42:46.442616 10811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:46.451000 10811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:46.453034 10811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:46.453187 10811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:46.457682 10811 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.252:34035
I20260807 06:42:46.457765 10901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.252:34035 every 8 connection(s)
I20260807 06:42:46.458113 10811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
I20260807 06:42:46.459088 10904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:46.460397 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 10811
I20260807 06:42:46.460520 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084684348861-11723-0/minicluster-data/master-2/wal/instance
I20260807 06:42:46.460757 11723 external_mini_cluster.cc:1428] Setting key ef80186c1a65ba9cf64faa155ff093d1
I20260807 06:42:46.461141 10904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:42:46.471037 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.462026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54970 (local address 127.11.114.254:46591)
0807 06:42:46.462075 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.462078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.462188 (+ 110us) server_negotiation.cc:407] Connection header received
0807 06:42:46.462464 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.462465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.462477 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.462546 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.463155 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.463623 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.464746 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.465040 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.465106 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.467639 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.467640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.467640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.467647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.469487 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.469968 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.469970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.469970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.470034 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.470291 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.470292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.470293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.470430 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.470515 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.470830 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.470924 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:42:46.471216 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.461904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:46591 (local address 127.0.0.1:54970)
0807 06:42:46.462051 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.462099 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.462350 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.462574 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.462577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.462860 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.463042 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.463046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.463701 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.463702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.464655 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.464657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.465137 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.465138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.465283 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.465896 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.465904 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.467533 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.469614 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.469621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.469622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.469879 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.470106 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.470106 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.470107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.470207 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.470505 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.470512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.470676 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.470805 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.470964 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260807 06:42:46.471597 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.461345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.11.114.252:34035)
0807 06:42:46.461623 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.461633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.461666 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:42:46.462348 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.462358 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.462385 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.462482 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.463149 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.464234 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.465125 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.465638 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.465696 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.468275 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.468284 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.468285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.468292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.470201 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.470541 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.470544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.470545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.470620 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.470853 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.470854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.470854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.470991 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.471365 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.471413 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.471487 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":61,"threads_started":1}
I20260807 06:42:46.471810 10904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:46.479489 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.472094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:41691 (local address 127.0.0.1:49916)
0807 06:42:46.472137 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.472147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.472189 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.472392 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.472393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.472577 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.472630 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.472632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.473510 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.473512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.474254 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.474256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.474739 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.474740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.474843 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.475212 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.475216 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.476545 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.478344 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.478345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.478346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.478531 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.478770 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.478771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.478771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.478838 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.479127 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.479128 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.479173 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.479280 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.479385 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 06:42:46.479499 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.472181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.11.114.253:41691)
0807 06:42:46.472257 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.472260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.472268 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:42:46.472295 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.472295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.472307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.472367 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.472716 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.473437 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.474328 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.474621 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.474713 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.476644 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.476645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.476646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.476655 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.478232 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.478623 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.478624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.478625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.478695 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.478915 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.478916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.478916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.479046 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.479135 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.479324 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.479404 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:42:46.480080 10904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:46.480400 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.472535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.11.114.254:46591)
0807 06:42:46.472580 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.472582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.472621 (+ 39us) server_negotiation.cc:407] Connection header received
0807 06:42:46.472665 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.472666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.472678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.472740 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.473276 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.473977 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.474828 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.475134 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.475216 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.477658 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.477659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.477660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.477667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.479070 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.479420 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.479421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.479422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.479480 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.479681 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.479682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.479682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.479815 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.479994 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.480202 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.480294 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:42:46.482671 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } attempt: 2
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:42:46.487648 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.480340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:50996)
0807 06:42:46.480374 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.480384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.480416 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.480719 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.480719 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.480880 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.480916 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.480918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.481612 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.481613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.482179 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.482181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.482578 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.482579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.482661 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.483030 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.483035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.484499 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.486406 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.486407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.486407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.486642 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.486860 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.486861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.486861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.486915 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.487263 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.487264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.487310 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.487393 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.487486 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 06:42:46.487749 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.480518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.11.114.252:34035)
0807 06:42:46.480561 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.480563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.480580 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:42:46.480597 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.480598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.480606 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.480683 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.481032 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.481489 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.482259 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.482503 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.482551 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.484608 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.484609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.484610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.484616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.486333 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.486738 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.486739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.486739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.486793 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.486998 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.486999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.486999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.487174 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.487261 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.487437 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.487549 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":23}
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:42:46.489586 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.481028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.11.114.253:41691)
0807 06:42:46.481074 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.481076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.481086 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:42:46.481146 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.481146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.481157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.481219 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.481787 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.482233 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.483251 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.483548 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.483605 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.485992 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.485993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.485994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.486003 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.488261 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.488662 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.488663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.488664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.488720 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.488954 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.488955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.488955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.489109 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.489314 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.489360 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.489442 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:42:46.489940 10904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f: Bootstrap starting.
I20260807 06:42:46.490806 10904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:46.491168 10904 log.cc:824] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:46.491879 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.483175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:51000)
0807 06:42:46.483232 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.483243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.483281 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.483547 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.483548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.483717 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.483758 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.483761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.484708 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.484709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.486225 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.486228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.486714 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.486716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.486776 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.487114 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.487119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.488634 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.490578 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.490579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.490580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.490756 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.490979 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.490980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.490981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.491040 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.491496 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.491497 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.491549 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.491628 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.491765 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 06:42:46.492089 10904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f: No bootstrap required, opened a new log
I20260807 06:42:46.492178 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:46.492319 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.483245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.11.114.252:34035)
0807 06:42:46.483418 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.483420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.483426 (+ 6us) server_negotiation.cc:407] Connection header received
0807 06:42:46.483442 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.483443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.483451 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.483512 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.483864 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.484618 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.486305 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.486604 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.486710 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.488757 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.488758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.488758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.488765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.490491 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.490833 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.490834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.490834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.490896 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.491195 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.491196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.491196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.491369 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.491564 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.492101 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.492189 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260807 06:42:46.494029 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159: Bootstrap starting.
I20260807 06:42:46.493338 10672 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54984
I20260807 06:42:46.494320 10904 raft_consensus.cc:348] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } }
I20260807 06:42:46.494486 10904 raft_consensus.cc:729] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec8226229a442738166068afa5b908f, State: Initialized, Role: FOLLOWER
I20260807 06:42:46.494597 10904 consensus_queue.cc:260] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } }
I20260807 06:42:46.494798 10706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:46.494992 10904 sys_catalog.cc:564] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:46.494940 10916 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.495102 10706 log.cc:824] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:46.495100 10916 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:46.496086 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159: No bootstrap required, opened a new log
W20260807 06:42:46.497462 10932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:42:46.497521 10932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:42:46.497671 10706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } }
I20260807 06:42:46.497820 10706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7f6e3d9f914ed4b8311b9e54bcf159, State: Initialized, Role: FOLLOWER
I20260807 06:42:46.497905 10706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } }
I20260807 06:42:46.498241 10706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:46.498200 10934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.498395 10934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:46.497370 10811 master_runner.cc:186] Error getting master registration for 127.11.114.254:46591: OK, instance_id { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" instance_seqno: 1786084966137898 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:42:46.500065 10811 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:42:46.500167 10672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 06:42:46.500248 10672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [term 0 FOLLOWER]: Advancing to term 1
W20260807 06:42:46.500680 10948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:42:46.500801 10948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:42:46.501168 10672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } }
I20260807 06:42:46.501520 10672 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [CANDIDATE]: Term 1 election: Requested vote from peers 44a545848a674792ad8e844c70837201 (127.11.114.253:41691), eec8226229a442738166068afa5b908f (127.11.114.252:34035)
I20260807 06:42:46.502265 10864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eec8226229a442738166068afa5b908f"
I20260807 06:42:46.502395 10864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:46.503363 10864 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d7f6e3d9f914ed4b8311b9e54bcf159 in term 1.
I20260807 06:42:46.503659 10635 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d7f6e3d9f914ed4b8311b9e54bcf159, eec8226229a442738166068afa5b908f; no voters:
I20260807 06:42:46.503777 10934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:46.503926 10934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [term 1 LEADER]: Becoming Leader. State: Replica: 3d7f6e3d9f914ed4b8311b9e54bcf159, State: Running, Role: LEADER
I20260807 06:42:46.504046 10934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } }
I20260807 06:42:46.504487 10934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7f6e3d9f914ed4b8311b9e54bcf159. Latest consensus state: current_term: 1 leader_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.504580 10934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:46.502722 10613 master_runner.cc:186] Error getting master registration for 127.11.114.253:41691: OK, instance_id { permanent_uuid: "44a545848a674792ad8e844c70837201" instance_seqno: 1786084966294232 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:42:46.508441 10613 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:42:46.513790 10962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:46.513867 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.503606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.11.114.252:34035)
0807 06:42:46.503641 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.503643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.504310 (+ 667us) server_negotiation.cc:407] Connection header received
0807 06:42:46.504454 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.504466 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.504478 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.504511 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.505622 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.506465 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.507927 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.508288 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.508335 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.510051 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.510053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.510053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.510061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.511895 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.512595 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.512597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.512597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.512673 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.513032 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.513033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.513033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.513191 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.513284 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.513682 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.513770 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:42:46.514647 10962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:42:46.514971 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.503292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.11.114.253:41691)
0807 06:42:46.503337 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.503339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.504027 (+ 688us) server_negotiation.cc:407] Connection header received
0807 06:42:46.504145 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.504147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.504160 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.504214 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.505037 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.505820 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.509049 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.509340 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.509386 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.511952 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.511954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.511954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.511963 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.513673 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.514084 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.514085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.514086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.514149 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.514369 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.514370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.514370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.514512 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.514578 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.514803 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.514884 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:42:46.516700 10864 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [term 1 FOLLOWER]: Refusing update from remote peer 3d7f6e3d9f914ed4b8311b9e54bcf159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:46.517077 10934 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:46.518431 10916 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7f6e3d9f914ed4b8311b9e54bcf159. Latest consensus state: current_term: 1 leader_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.518579 10916 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:46.520984 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.503588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.11.114.254:46591)
0807 06:42:46.504753 (+ 1165us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.504755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.504765 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:42:46.504787 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.504788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.504799 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.504866 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.505889 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.506566 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.507594 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.507873 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.507944 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.517385 (+ 9441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.517387 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.517387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.517425 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.519691 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.520087 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.520089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.520089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.520159 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.520380 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.520381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.520382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.520533 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.520655 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.520810 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.520895 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1144}
I20260807 06:42:46.530560 10934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.530661 10934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:46.531180 10916 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.531270 10916 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:46.531773 10962 catalog_manager.cc:1347] Generated new cluster ID: 2d5038bc4fc041ce8a4cf4575a18842e
I20260807 06:42:46.531831 10962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:46.533051 10803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } attempt: 3
I20260807 06:42:46.542043 10962 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:46.542753 10962 catalog_manager.cc:1504] Loading token signing keys...
Aug 07 06:42:46 dist-test-slave-gkw7 krb5kdc[10595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:42:46.548221 10962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159: Generated new TSK 0
I20260807 06:42:46.548410 10962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:46.549265 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.533267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:34035 (local address 127.0.0.1:51016)
0807 06:42:46.533307 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:46.533316 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:46.533346 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:46.540850 (+ 7504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:46.540851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:46.541039 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:46.541089 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.541091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.542064 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.542065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.542873 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:46.542875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.543388 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.543389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:46.543444 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.543923 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:46.543929 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:46.546014 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:46.547949 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.547950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.547951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.548194 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.548542 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:46.548543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:46.548544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:46.548639 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:46.548927 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:46.548928 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:46.548976 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 06:42:46.549042 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:42:46.549158 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:42:46.549589 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.540641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.11.114.252:34035)
0807 06:42:46.540688 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.540690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.540700 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:42:46.540721 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.540722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.540735 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.540796 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.541184 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.541972 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.542969 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.543275 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.543334 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.546094 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:46.546095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:46.546095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:46.546102 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:46.547844 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.548280 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.548282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.548282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.548340 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:46.548718 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:46.548719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:46.548720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:46.548836 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:46.548927 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.549386 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.549479 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:42:46.551210 10803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201: Bootstrap starting.
I20260807 06:42:46.551975 10803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:46.552309 10803 log.cc:824] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:46.553272 10803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201: No bootstrap required, opened a new log
I20260807 06:42:46.555437 10803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } }
I20260807 06:42:46.555579 10803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a545848a674792ad8e844c70837201, State: Initialized, Role: FOLLOWER
I20260807 06:42:46.555665 10803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } }
I20260807 06:42:46.556022 10803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:46.556147 10767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "44a545848a674792ad8e844c70837201"
I20260807 06:42:46.556254 10767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:46.557185 10767 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d7f6e3d9f914ed4b8311b9e54bcf159 in term 1.
I20260807 06:42:46.557977 10766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [term 1 FOLLOWER]: Refusing update from remote peer 3d7f6e3d9f914ed4b8311b9e54bcf159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:46.561141 10960 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:46.563966 10988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.564073 10988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:46.564136 10988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.564191 10988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:46.566500 10991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7f6e3d9f914ed4b8311b9e54bcf159. Latest consensus state: current_term: 1 leader_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:46.566612 10991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:46.568346 10985 catalog_manager.cc:1259] Loaded cluster ID: 2d5038bc4fc041ce8a4cf4575a18842e
I20260807 06:42:46.568394 10985 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201: loading cluster ID for follower catalog manager: success
I20260807 06:42:46.569566 10985 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201: acquiring CA information for follower catalog manager: success
I20260807 06:42:46.569887 10985 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 44a545848a674792ad8e844c70837201: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 06:42:46.587018 10808 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.585293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.11.114.253:41691)
0807 06:42:46.585340 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.585342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.586334 (+ 992us) server_negotiation.cc:407] Connection header received
0807 06:42:46.586441 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.586442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.586453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.586485 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.586950 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49956: BlockingRecv error: recv got EOF from 127.0.0.1:49956 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 06:42:46.587803 10711 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.585768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.11.114.254:46591)
0807 06:42:46.585811 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.585813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.587274 (+ 1461us) server_negotiation.cc:407] Connection header received
0807 06:42:46.587375 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.587376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.587388 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.587417 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.587738 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54998: BlockingRecv error: recv got EOF from 127.0.0.1:54998 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 06:42:46.588522 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.586076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.11.114.252:34035)
0807 06:42:46.586117 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.586119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.587998 (+ 1879us) server_negotiation.cc:407] Connection header received
0807 06:42:46.588125 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.588126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.588137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.588166 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.588462 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51022: BlockingRecv error: recv got EOF from 127.0.0.1:51022 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:42:46.593674 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.589786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.11.114.253:41691)
0807 06:42:46.589819 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.589821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.590381 (+ 560us) server_negotiation.cc:407] Connection header received
0807 06:42:46.590501 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.590502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.590539 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.590582 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.590796 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.591586 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.592704 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.593000 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.593038 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.593262 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.593389 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.593437 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.593575 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.593595 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 06:42:46.596946 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.596168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.11.114.252:34035)
0807 06:42:46.596205 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.596207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.596378 (+ 171us) server_negotiation.cc:407] Connection header received
0807 06:42:46.596484 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.596485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.596504 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.596551 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.596897 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51028: BlockingRecv error: recv got EOF from 127.0.0.1:51028 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:42:46.602438 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.599181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.11.114.253:41691)
0807 06:42:46.599222 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.599224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.599461 (+ 237us) server_negotiation.cc:407] Connection header received
0807 06:42:46.599550 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.599551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.599560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.599587 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.599774 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.600488 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.601645 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.601963 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.602004 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.602198 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.602230 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.602257 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.602358 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.602377 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 06:42:46.606051 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.605266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.11.114.252:34035)
0807 06:42:46.605303 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.605305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.605494 (+ 189us) server_negotiation.cc:407] Connection header received
0807 06:42:46.605624 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.605625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.605636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.605664 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.606000 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51038: BlockingRecv error: recv got EOF from 127.0.0.1:51038 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:42:46.612377 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.609131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.11.114.253:41691)
0807 06:42:46.609165 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.609167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.609400 (+ 233us) server_negotiation.cc:407] Connection header received
0807 06:42:46.609490 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.609491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.609501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.609627 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.609796 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.610546 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.611635 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.611920 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.611956 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.612140 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.612172 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.612197 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.612294 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.612311 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 06:42:46.615319 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.614597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.11.114.252:34035)
0807 06:42:46.614644 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.614646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.614812 (+ 166us) server_negotiation.cc:407] Connection header received
0807 06:42:46.614901 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.614902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.614912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.614940 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.615273 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51044: BlockingRecv error: recv got EOF from 127.0.0.1:51044 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 06:42:46.623713 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.620388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49994 (local address 127.11.114.253:41691)
0807 06:42:46.620421 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.620423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.620664 (+ 241us) server_negotiation.cc:407] Connection header received
0807 06:42:46.620754 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.620755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.620764 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.620791 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.620943 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.621674 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.622883 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.623195 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.623235 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.623436 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.623468 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.623496 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.623593 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.623646 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 06:42:46.626705 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.625969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.11.114.252:34035)
0807 06:42:46.626010 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.626012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.626182 (+ 170us) server_negotiation.cc:407] Connection header received
0807 06:42:46.626279 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.626280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.626291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.626321 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.626657 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51052: BlockingRecv error: recv got EOF from 127.0.0.1:51052 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:42:46.639204 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.635902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.11.114.253:41691)
0807 06:42:46.635944 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.635946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.636215 (+ 269us) server_negotiation.cc:407] Connection header received
0807 06:42:46.636313 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.636314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.636325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.636353 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.636522 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.637269 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.638414 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.638720 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.638760 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.638953 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.638987 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.639013 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.639112 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.639131 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 06:42:46.642194 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.641439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.11.114.252:34035)
0807 06:42:46.641477 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.641479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.641665 (+ 186us) server_negotiation.cc:407] Connection header received
0807 06:42:46.641754 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.641755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.641766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.641812 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.642142 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51060: BlockingRecv error: recv got EOF from 127.0.0.1:51060 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:42:46.663213 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.659531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.11.114.253:41691)
0807 06:42:46.659577 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.659579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.659919 (+ 340us) server_negotiation.cc:407] Connection header received
0807 06:42:46.660030 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.660031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.660045 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.660077 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.660434 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.661172 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.662386 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.662681 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.662728 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.662941 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.662978 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.663006 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.663114 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.663133 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 06:42:46.666544 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.665734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.11.114.252:34035)
0807 06:42:46.665781 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.665783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.665975 (+ 192us) server_negotiation.cc:407] Connection header received
0807 06:42:46.666074 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.666075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.666088 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.666119 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.666491 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51064: BlockingRecv error: recv got EOF from 127.0.0.1:51064 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:42:46.703588 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.700050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50018 (local address 127.11.114.253:41691)
0807 06:42:46.700098 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.700100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.700423 (+ 323us) server_negotiation.cc:407] Connection header received
0807 06:42:46.700534 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.700535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.700547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.700578 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.700809 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.701593 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.702774 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.703062 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.703107 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.703318 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.703353 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.703380 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.703483 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.703504 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 06:42:46.706959 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.706114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.11.114.252:34035)
0807 06:42:46.706156 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.706158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.706364 (+ 206us) server_negotiation.cc:407] Connection header received
0807 06:42:46.706425 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.706426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.706441 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.706524 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.706901 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51066: BlockingRecv error: recv got EOF from 127.0.0.1:51066 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:42:46.774451 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.772069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.11.114.253:41691)
0807 06:42:46.772151 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.772154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.772298 (+ 144us) server_negotiation.cc:407] Connection header received
0807 06:42:46.772335 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.772336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.772347 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.772416 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.772529 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.773035 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.773808 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.774006 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.774058 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.774207 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.774234 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.774270 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.774337 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.774353 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 06:42:46.777097 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.776354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.11.114.252:34035)
0807 06:42:46.776434 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.776436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.776464 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:42:46.776548 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.776549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.776560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.776638 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.777035 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51068: BlockingRecv error: recv got EOF from 127.0.0.1:51068 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:42:46.908762 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:46.906482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50040 (local address 127.11.114.253:41691)
0807 06:42:46.906548 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.906551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.906643 (+ 92us) server_negotiation.cc:407] Connection header received
0807 06:42:46.906709 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.906709 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.906719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.906784 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:46.906895 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.907366 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.908117 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:46.908300 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:46.908355 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:46.908493 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:46.908521 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:46.908558 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:46.908622 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 06:42:46.908638 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 06:42:46.911105 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:46.910509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.11.114.252:34035)
0807 06:42:46.910555 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:46.910557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:46.910621 (+ 64us) server_negotiation.cc:407] Connection header received
0807 06:42:46.910677 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:46.910678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:46.910688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:46.910734 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:46.911055 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51072: BlockingRecv error: recv got EOF from 127.0.0.1:51072 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:42:47.112586 10960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7f6e3d9f914ed4b8311b9e54bcf159" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 46591 } } peers { permanent_uuid: "44a545848a674792ad8e844c70837201" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 41691 } } peers { permanent_uuid: "eec8226229a442738166068afa5b908f" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 34035 } } }
I20260807 06:42:47.112756 10960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f6e3d9f914ed4b8311b9e54bcf159 [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:47.170723 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:47.168498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50048 (local address 127.11.114.253:41691)
0807 06:42:47.168568 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:47.168570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:47.168623 (+ 53us) server_negotiation.cc:407] Connection header received
0807 06:42:47.168690 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:47.168691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:47.168700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:47.168765 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:47.168887 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.169367 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.170116 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.170297 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.170348 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:47.170486 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:47.170513 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:47.170551 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:47.170604 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 06:42:47.170620 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 06:42:47.173115 10913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:47.172555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.11.114.252:34035)
0807 06:42:47.172617 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:47.172619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:47.172628 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:47.172659 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:47.172659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:47.172669 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:47.172718 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:47.173075 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51082: BlockingRecv error: recv got EOF from 127.0.0.1:51082 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:42:47.498014 10932 catalog_manager.cc:1259] Loaded cluster ID: 2d5038bc4fc041ce8a4cf4575a18842e
I20260807 06:42:47.498095 10932 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f: loading cluster ID for follower catalog manager: success
I20260807 06:42:47.499114 10932 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f: acquiring CA information for follower catalog manager: success
I20260807 06:42:47.499464 10932 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eec8226229a442738166068afa5b908f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:42:47.688850 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:47.686527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.11.114.253:41691)
0807 06:42:47.686630 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:47.686632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:47.686643 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:42:47.686715 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:47.686716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:47.686726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:47.686803 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:47.686941 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.687440 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.688162 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.688360 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.688413 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:47.688614 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:47.688643 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:47.688677 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:47.688765 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 06:42:47.688784 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":56,"threads_started":1}
I20260807 06:42:47.693218 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:47.690731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51094 (local address 127.11.114.252:34035)
0807 06:42:47.690843 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:47.690845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:47.690863 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:42:47.690888 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:47.690889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:47.690939 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:47.691013 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:47.691107 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.691659 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.692427 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.692629 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.692698 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:47.692833 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:47.692945 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:47.693005 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:47.693041 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 06:42:47.693061 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260807 06:42:47.698222 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:47.695051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.11.114.253:41691)
0807 06:42:47.695117 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:47.695120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:47.695208 (+ 88us) server_negotiation.cc:407] Connection header received
0807 06:42:47.695343 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:47.695344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:47.695354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:47.695386 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:47.695556 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.696267 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.697290 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.697601 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.697679 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:47.697817 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:47.697849 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:47.697899 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:47.698090 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 06:42:47.698115 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:42:47.698496 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:47.695006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.11.114.254:46591)
0807 06:42:47.695122 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:47.695161 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:47.695329 (+ 168us) server_negotiation.cc:407] Connection header received
0807 06:42:47.695381 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:47.695382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:47.695440 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:47.695530 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:47.695709 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.696532 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.697481 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.697824 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.697894 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:47.698059 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:47.698229 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:47.698284 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:47.698386 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 06:42:47.698411 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":69,"threads_started":1}
I20260807 06:42:47.698841 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:47.695299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.11.114.252:34035)
0807 06:42:47.695334 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:47.695336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:47.695812 (+ 476us) server_negotiation.cc:407] Connection header received
0807 06:42:47.695836 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:47.695837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:47.695848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:47.695896 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:47.696036 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.696712 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.697707 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:47.698024 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:47.698102 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:47.698602 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:47.698639 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:47.698681 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:47.698744 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 06:42:47.698767 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:42:47.705834 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 10613
I20260807 06:42:47.722586 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 10715
I20260807 06:42:47.731025 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 10811
2026-08-07T06:42:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3160 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:42:47 dist-test-slave-gkw7 krb5kdc[11171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:42:47 dist-test-slave-gkw7 krb5kdc[11171](info): set up 2 sockets
Aug 07 06:42:47 dist-test-slave-gkw7 krb5kdc[11171](info): commencing operation
krb5kdc: starting...
W20260807 06:42:48.943377 11723 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/test-user.keytab.
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:42:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.114.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
Entry for principal kudu/127.11.114.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab.
I20260807 06:42:49.069126 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40649
--webserver_interface=127.11.114.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:40649,127.11.114.253:38811,127.11.114.252:36567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:42:49.151871 11190 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:42:49.152081 11190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:49.152099 11190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:49.152118 11190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:49.153403 11190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:49.153443 11190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:49.153458 11190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:49.153471 11190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:49.153518 11190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:49.155094 11190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:40649,127.11.114.253:38811,127.11.114.252:36567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.254.keytab
--principal=kudu/127.11.114.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.254:40649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.114.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:49.155361 11190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:49.155612 11190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:49.160622 11197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:49.160632 11202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:49.160714 11198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:49.160849 11190 server_base.cc:1034] running on GCE node
I20260807 06:42:49.161141 11190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:49.161396 11190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:49.162541 11190 hybrid_clock.cc:648] HybridClock initialized: now 1786084969162519 us; error 44 us; skew 500 ppm
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:49.165011 11190 init.cc:377] Logged in from keytab as kudu/127.11.114.254@KRBTEST.COM (short username kudu)
I20260807 06:42:49.165438 11190 webserver.cc:460] Webserver started at http://127.11.114.254:34993/ using document root <none> and password file <none>
I20260807 06:42:49.165727 11190 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:49.165776 11190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:49.165923 11190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:49.166975 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/data/instance:
uuid: "5a2dc5d6191043c9bef0f4da078d0c7f"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "8227c3c70ffc4a99994ed6bfa926d97f"
server_key_iv: "ab0234f271c1206a326cadbd244666a8"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.167349 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/wal/instance:
uuid: "5a2dc5d6191043c9bef0f4da078d0c7f"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "8227c3c70ffc4a99994ed6bfa926d97f"
server_key_iv: "ab0234f271c1206a326cadbd244666a8"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.168799 11190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:49.169631 11217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:49.169862 11190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:49.169991 11190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/wal
uuid: "5a2dc5d6191043c9bef0f4da078d0c7f"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "8227c3c70ffc4a99994ed6bfa926d97f"
server_key_iv: "ab0234f271c1206a326cadbd244666a8"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.170099 11190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:49.193388 11190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:49.196287 11190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:49.196491 11190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:49.201851 11190 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.254:40649
I20260807 06:42:49.201906 11289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.254:40649 every 8 connection(s)
I20260807 06:42:49.202333 11190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/data/info.pb
I20260807 06:42:49.203351 11291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:49.204397 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 11190
I20260807 06:42:49.204480 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-0/wal/instance
I20260807 06:42:49.204708 11723 external_mini_cluster.cc:1428] Setting key a80de9ed25d660b3b364fc95830cf355
I20260807 06:42:49.205142 11291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.114.253@KRBTEST.COM; defaulting to no policy
I20260807 06:42:49.216292 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.206124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.11.114.254:40649)
0807 06:42:49.206277 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.206281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.206314 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:42:49.206843 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.206845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.206865 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.206971 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.207551 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.208389 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.209721 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.210319 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.210453 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.212858 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.212866 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.212866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.212873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.214629 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.215035 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.215038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.215038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.215129 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.215363 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.215364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.215365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.215593 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.215685 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.215898 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.216039 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260807 06:42:49.216295 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.205911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40649 (local address 127.0.0.1:54072)
0807 06:42:49.206112 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.206148 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.206679 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.206991 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.206993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.207281 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.207430 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.207434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.208492 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.208493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.209629 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.209632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.210425 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.210427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.210523 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.211173 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.211184 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.212752 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.214731 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.214738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.214740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.214937 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.215199 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.215200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.215200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.215295 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.215675 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.215680 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.215746 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.215850 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.216113 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":110,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.253@KRBTEST.COM" created.
I20260807 06:42:49.217125 11291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:49.217480 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.217362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51330)
0807 06:42:49.217410 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.217456 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:42:49.217641 11219 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
I20260807 06:42:49.218016 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.217952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51332)
0807 06:42:49.217976 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.217999 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260807 06:42:49.218142 11291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:38811: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab.
Entry for principal kudu/127.11.114.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab.
I20260807 06:42:49.232394 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.253:38811
--webserver_interface=127.11.114.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab
--principal=kudu/127.11.114.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:40649,127.11.114.253:38811,127.11.114.252:36567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:42:49.267379 11291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } attempt: 1
I20260807 06:42:49.267927 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.267784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51348)
0807 06:42:49.267853 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.267896 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 06:42:49.268365 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.268227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51354)
0807 06:42:49.268301 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.268334 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 06:42:49.268502 11291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:38811: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
W20260807 06:42:49.318094 11299 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:42:49.318295 11299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:49.318310 11299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:49.318331 11299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:49.319410 11299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:49.319444 11299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:49.319458 11299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:49.319471 11299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:49.319483 11299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:49.320865 11299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:40649,127.11.114.253:38811,127.11.114.252:36567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.253.keytab
--principal=kudu/127.11.114.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.253:38811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.114.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:49.321062 11299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:49.321228 11299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:49.326210 11306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:49.326304 11310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:49.326236 11312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:49.326474 11299 server_base.cc:1034] running on GCE node
I20260807 06:42:49.326713 11299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:49.327001 11299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:49.328210 11299 hybrid_clock.cc:648] HybridClock initialized: now 1786084969328188 us; error 33 us; skew 500 ppm
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:49.330919 11299 init.cc:377] Logged in from keytab as kudu/127.11.114.253@KRBTEST.COM (short username kudu)
I20260807 06:42:49.331393 11299 webserver.cc:460] Webserver started at http://127.11.114.253:36669/ using document root <none> and password file <none>
I20260807 06:42:49.331622 11299 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:49.331668 11299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:49.331815 11299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:49.332883 11299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/data/instance:
uuid: "69ce0b647d3d4f658362c4835b0e243e"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "ea922e8e3560175befbaf07136cbe313"
server_key_iv: "d6803f183b821c2651d4fe6d5bd8e48f"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.333266 11299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/wal/instance:
uuid: "69ce0b647d3d4f658362c4835b0e243e"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "ea922e8e3560175befbaf07136cbe313"
server_key_iv: "d6803f183b821c2651d4fe6d5bd8e48f"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.334717 11299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:42:49.335572 11320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:49.335783 11299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:42:49.335872 11299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/wal
uuid: "69ce0b647d3d4f658362c4835b0e243e"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "ea922e8e3560175befbaf07136cbe313"
server_key_iv: "d6803f183b821c2651d4fe6d5bd8e48f"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.335928 11299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:49.336648 11291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } attempt: 2
I20260807 06:42:49.337265 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.337116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51356)
0807 06:42:49.337185 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.337220 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 06:42:49.337726 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.337628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51368)
0807 06:42:49.337678 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.337708 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:42:49.337895 11291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.253:38811: Network error: Client connection negotiation failed: client connection to 127.11.114.253:38811: connect: Connection refused (error 111)
I20260807 06:42:49.352864 11299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:49.355549 11299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:49.355746 11299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:49.360172 11299 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.253:38811
I20260807 06:42:49.360203 11399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.253:38811 every 8 connection(s)
I20260807 06:42:49.360590 11299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/data/info.pb
I20260807 06:42:49.361469 11400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:49.363076 11400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:49.367565 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 11299
I20260807 06:42:49.367714 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-1/wal/instance
I20260807 06:42:49.367933 11723 external_mini_cluster.cc:1428] Setting key c0b804a41f4a3d71c590da5b1ce1c939
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
I20260807 06:42:49.373193 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.363804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.11.114.254:40649)
0807 06:42:49.363877 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.363880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.364110 (+ 230us) server_negotiation.cc:407] Connection header received
0807 06:42:49.364614 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.364616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.364629 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.364678 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.365342 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.365935 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.366980 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.367197 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.367256 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.369870 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.369871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.369872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.369880 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.371503 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.371955 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.371956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.371957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.372050 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.372439 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.372440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.372440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.372572 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.372634 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.372984 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.373073 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:42:49.373358 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.363764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40649 (local address 127.0.0.1:54076)
0807 06:42:49.363959 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.363998 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.364496 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.364755 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.364759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.365027 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.365249 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.365253 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.366036 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.366037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.366893 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.366896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.367283 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.367284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.367423 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.367997 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.368006 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.369768 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.371623 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.371630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.371632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.371870 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.372162 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.372163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.372164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.372328 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.372648 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.372657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.372811 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.372958 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.373127 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260807 06:42:49.373993 11400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.114.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.114.252@KRBTEST.COM" created.
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:42:49.384464 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.374286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51384)
0807 06:42:49.374338 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.374349 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.374390 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.374699 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.374700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.374812 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.374860 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.374861 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.376337 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.376338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.376861 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.376862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.377644 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.377644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.377734 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.378170 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.378175 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.381288 (+ 3113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.383400 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.383401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.383401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.383578 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.383812 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.383812 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.383812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.383860 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.384216 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.384217 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.384256 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.384316 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.384398 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:42:49.384645 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.374346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51384 (local address 127.11.114.253:38811)
0807 06:42:49.374523 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.374528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.374554 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:42:49.374580 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.374583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.374601 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.374699 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.374938 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.376228 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.376950 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.377495 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.377637 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.381400 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.381407 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.381408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.381414 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.383325 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.383667 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.383671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.383671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.383742 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.383924 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.383925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.383925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.384138 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.384230 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.384374 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.384481 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260807 06:42:49.385519 11400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:49.385984 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.385829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42686)
0807 06:42:49.385888 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.385948 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 06:42:49.386214 11323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
I20260807 06:42:49.386693 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.386569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42698)
0807 06:42:49.386625 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.386661 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:42:49.386827 11400 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:36567: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.114.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab.
Entry for principal kudu/127.11.114.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab.
I20260807 06:42:49.396549 11723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZbZY4B/build/release/bin/kudu
/tmp/dist-test-taskZbZY4B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.252:36567
--webserver_interface=127.11.114.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab
--principal=kudu/127.11.114.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.114.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.114.254:40649,127.11.114.253:38811,127.11.114.252:36567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:42:49.429092 11291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } attempt: 3
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:42:49.436077 11400 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } attempt: 1
I20260807 06:42:49.436530 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.436400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42714)
0807 06:42:49.436460 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.436490 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 06:42:49.437018 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.436901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42716)
0807 06:42:49.436939 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.436964 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 06:42:49.437124 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.429631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.11.114.253:38811)
0807 06:42:49.429703 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.429705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.429717 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:42:49.429786 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.429788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.429808 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.429882 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.430290 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.430789 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.431559 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.431793 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.431877 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.434092 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.434094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.434094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.434102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.435698 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.436143 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.436144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.436144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.436219 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.436460 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.436461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.436461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.436590 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.436681 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.436875 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.436962 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:42:49.437127 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.429557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51392)
0807 06:42:49.429625 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.429643 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.429715 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.429889 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.429891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.430071 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.430179 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.430182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.430918 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.430920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.431460 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.431462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.431878 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.431879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.431970 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.432496 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.432502 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.433987 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.435817 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.435819 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.435819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.436043 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.436320 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.436321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.436321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.436388 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.436684 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.436686 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.436746 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.436830 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.437012 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 06:42:49.437422 11400 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:36567: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
I20260807 06:42:49.437685 11291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:49.438007 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.437923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42728)
0807 06:42:49.437962 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.437988 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:42:49.438315 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.438225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42744)
0807 06:42:49.438273 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.438296 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:42:49.438412 11291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:36567: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
I20260807 06:42:49.469554 11291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } attempt: 1
I20260807 06:42:49.470026 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.469891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42758)
0807 06:42:49.469956 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.469996 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 06:42:49.470417 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.470317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42766)
0807 06:42:49.470365 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.470391 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:42:49.470520 11291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:36567: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
W20260807 06:42:49.481235 11407 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:42:49.481480 11407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:42:49.481515 11407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:42:49.481578 11407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:42:49.482751 11407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:42:49.482817 11407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:42:49.482841 11407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:42:49.482868 11407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:42:49.482892 11407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:42:49.484377 11407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.114.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.114.254:40649,127.11.114.253:38811,127.11.114.252:36567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/krb5kdc/kudu_127.11.114.252.keytab
--principal=kudu/127.11.114.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.114.252:36567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.114.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:42:49.484663 11407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:42:49.484880 11407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:42:49.489861 11416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:42:49.489850 11414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:49.489867 11407 server_base.cc:1034] running on GCE node
W20260807 06:42:49.489850 11413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:42:49.490278 11407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:42:49.490540 11407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:42:49.491691 11407 hybrid_clock.cc:648] HybridClock initialized: now 1786084969491675 us; error 37 us; skew 500 ppm
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:42:49.494215 11407 init.cc:377] Logged in from keytab as kudu/127.11.114.252@KRBTEST.COM (short username kudu)
I20260807 06:42:49.494644 11407 webserver.cc:460] Webserver started at http://127.11.114.252:37173/ using document root <none> and password file <none>
I20260807 06:42:49.494865 11407 fs_manager.cc:359] Metadata directory not provided
I20260807 06:42:49.494932 11407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:42:49.495081 11407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:42:49.496146 11407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/data/instance:
uuid: "114acace0b06464ebbf109656386cf98"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "6faca4829471bcab13d2dc75872727c4"
server_key_iv: "5d074cc0245c6e3912264744e0a653ee"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.496521 11407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/wal/instance:
uuid: "114acace0b06464ebbf109656386cf98"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "6faca4829471bcab13d2dc75872727c4"
server_key_iv: "5d074cc0245c6e3912264744e0a653ee"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.497944 11407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:42:49.498703 11424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:42:49.498879 11407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:42:49.498984 11407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/data,/tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/wal
uuid: "114acace0b06464ebbf109656386cf98"
format_stamp: "Formatted at 2026-08-07 06:42:49 on dist-test-slave-gkw7"
server_key: "6faca4829471bcab13d2dc75872727c4"
server_key_iv: "5d074cc0245c6e3912264744e0a653ee"
server_key_version: "encryptionkey@0"
I20260807 06:42:49.499058 11407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:42:49.505591 11400 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } attempt: 2
I20260807 06:42:49.506053 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.505918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42768)
0807 06:42:49.505985 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.506023 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 06:42:49.506364 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.506281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42778)
0807 06:42:49.506325 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.506349 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:42:49.506438 11400 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.114.252:36567: Network error: Client connection negotiation failed: client connection to 127.11.114.252:36567: connect: Connection refused (error 111)
I20260807 06:42:49.522017 11407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:42:49.523968 11407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:42:49.524075 11407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:42:49.528007 11407 rpc_server.cc:307] RPC server started. Bound to: 127.11.114.252:36567
I20260807 06:42:49.528067 11495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.114.252:36567 every 8 connection(s)
I20260807 06:42:49.528424 11407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/data/info.pb
I20260807 06:42:49.529703 11496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:42:49.531301 11496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:49.531653 11723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZbZY4B/build/release/bin/kudu as pid 11407
I20260807 06:42:49.531761 11723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZbZY4B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084684348861-11723-0/minicluster-data/master-2/wal/instance
I20260807 06:42:49.532022 11723 external_mini_cluster.cc:1428] Setting key 45868ea8be5b968139f8f65fad0d0dee
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:42:49.541846 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.532065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.11.114.254:40649)
0807 06:42:49.532119 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.532121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.532365 (+ 244us) server_negotiation.cc:407] Connection header received
0807 06:42:49.532983 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.532985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.532996 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.533087 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.533834 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.534644 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.535841 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.536123 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.536201 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.538519 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.538521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.538522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.538529 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.540284 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.540770 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.540771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.540771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.540828 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.541108 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.541109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.541109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.541225 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.541278 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.541649 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.541735 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:42:49.542026 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.532046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.254:40649 (local address 127.0.0.1:54082)
0807 06:42:49.532229 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.532280 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.532878 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.533101 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.533103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.533478 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.533737 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.533742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.534727 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.534728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.535737 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.535740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.536221 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.536222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.536322 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.536814 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.536825 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.538427 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.540394 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.540399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.540401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.540692 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.540896 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.540897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.540898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.541033 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.541294 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.541302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.541464 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.541623 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.541770 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260807 06:42:49.542577 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.532446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.11.114.252:36567)
0807 06:42:49.532626 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.532630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.532686 (+ 56us) server_negotiation.cc:407] Connection header received
0807 06:42:49.532889 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.532900 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.532922 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.533035 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.533841 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.534947 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.535984 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.536402 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.536456 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.539074 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.539080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.539081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.539087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.541052 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.541452 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.541455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.541455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.541517 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.541819 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.541820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.541820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.541961 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.542324 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.542388 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.542481 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260807 06:42:49.542727 11496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:49.545636 11291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } attempt: 2
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.254@KRBTEST.COM
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.254@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:42:49.550577 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.543049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.11.114.253:38811)
0807 06:42:49.543090 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.543092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.543100 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:42:49.543135 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.543136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.543145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.543191 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.543518 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.543985 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.544644 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.544867 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.544943 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.546960 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.546962 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.546962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.546969 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.549282 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.549713 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.549715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.549715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.549780 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.549981 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.549982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.549982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.550129 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.550193 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.550403 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.550485 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:42:49.550882 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.542994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.253:38811 (local address 127.0.0.1:51396)
0807 06:42:49.543032 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.543042 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.543076 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.543203 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.543203 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.543360 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.543419 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.543421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.544068 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.544069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.544544 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.544545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.544939 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.544940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.545008 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.545416 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.545421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.546871 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.549372 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.549373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.549374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.549630 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.549849 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.549849 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.549850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.549908 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.550223 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.550224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.550279 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.550697 (+ 418us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.550789 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 06:42:49.550953 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.543427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.11.114.254:40649)
0807 06:42:49.543470 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.543472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.543480 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:42:49.543499 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.543500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.543509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.543557 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.544016 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.544458 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.545121 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.545323 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.545407 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.547926 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.547927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.547927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.547935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.549716 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.550046 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.550047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.550048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.550113 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.550305 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.550306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.550306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.550439 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.550616 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.550661 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.550744 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:42:49.551227 11496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:42:49.553817 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.546061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42786 (local address 127.11.114.252:36567)
0807 06:42:49.546166 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.546169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.546183 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:42:49.546205 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.546206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.546217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.546284 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.546582 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.547331 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.548045 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.548361 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.548424 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.550460 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.550461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.550461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.550468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.552647 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.552986 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.552987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.552987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.553043 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.553259 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.553260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.553261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.553369 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.553443 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.553656 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.553732 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 06:42:49.553821 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.545860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42786)
0807 06:42:49.545946 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.545955 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.545993 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.546310 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.546311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.546464 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.546501 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.546502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.547404 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.547406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.547951 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.547953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.548423 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.548424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.548485 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.548813 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.548818 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.550378 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.552730 (+ 2352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.552731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.552731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.552910 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.553125 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.553126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.553126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.553185 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.553437 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.553438 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.553481 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.553627 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.553732 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 06:42:49.555819 11291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f: Bootstrap starting.
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.252@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:42:49.556595 11291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f: Neither blocks nor log segments found. Creating new log.
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.114.253@KRBTEST.COM
I20260807 06:42:49.556916 11291 log.cc:824] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:49.558012 11291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f: No bootstrap required, opened a new log
I20260807 06:42:49.559671 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.551500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42800)
0807 06:42:49.551528 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.551538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.551575 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.551904 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.551905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.552095 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.552121 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.552123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.552894 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.552894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.553619 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.553621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.554442 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.554443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.554484 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.554887 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.554892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.556317 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.558400 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.558401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.558402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.558617 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.558828 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.558829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.558829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.558903 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.559249 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.559250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.559300 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.559376 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.559485 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260807 06:42:49.559747 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.551532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42800 (local address 127.11.114.252:36567)
0807 06:42:49.551678 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.551680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.551687 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:42:49.551703 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.551704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.551712 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.551755 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.552196 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.552813 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.553695 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.554345 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.554402 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.556398 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.556400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.556400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.556408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.558292 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.558698 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.558699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.558699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.558749 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.558986 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.558987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.558987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.559171 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.559243 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.559405 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.559526 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260807 06:42:49.560480 11291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } }
I20260807 06:42:49.560637 11291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a2dc5d6191043c9bef0f4da078d0c7f, State: Initialized, Role: FOLLOWER
I20260807 06:42:49.560732 11291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } }
I20260807 06:42:49.561038 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.561102 11291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:49.561147 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:49.561194 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98: Bootstrap starting.
I20260807 06:42:49.561398 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.551585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.11.114.253:38811)
0807 06:42:49.551621 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.551623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.551742 (+ 119us) server_negotiation.cc:407] Connection header received
0807 06:42:49.551840 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.551841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.551853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.551916 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.552556 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.553226 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.554214 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.554510 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.554553 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.557423 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.557424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.557424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.557431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.559164 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.560289 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.560290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.560291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.560350 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.560586 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.560587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.560587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.560722 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.560912 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.561216 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.561310 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:42:49.561898 11496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:49.562232 11496 log.cc:824] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:49.563100 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98: No bootstrap required, opened a new log
W20260807 06:42:49.563292 11528 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:42:49.563340 11528 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:42:49.563468 11723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:42:49.564385 11252 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54096
I20260807 06:42:49.564514 11252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 06:42:49.564555 11252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:49.565249 11496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } }
I20260807 06:42:49.565385 11496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114acace0b06464ebbf109656386cf98, State: Initialized, Role: FOLLOWER
I20260807 06:42:49.565398 11252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } }
I20260807 06:42:49.565457 11496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } }
I20260807 06:42:49.565660 11252 leader_election.cc:288] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [CANDIDATE]: Term 1 election: Requested vote from peers 69ce0b647d3d4f658362c4835b0e243e (127.11.114.253:38811), 114acace0b06464ebbf109656386cf98 (127.11.114.252:36567)
I20260807 06:42:49.565795 11496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:49.563829 11190 master_runner.cc:186] Error getting master registration for 127.11.114.253:38811: OK, instance_id { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" instance_seqno: 1786084969358925 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:42:49.566236 11190 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:42:49.566227 11454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "114acace0b06464ebbf109656386cf98"
I20260807 06:42:49.566344 11454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:49.567152 11454 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a2dc5d6191043c9bef0f4da078d0c7f in term 1.
I20260807 06:42:49.567440 11219 leader_election.cc:302] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114acace0b06464ebbf109656386cf98, 5a2dc5d6191043c9bef0f4da078d0c7f; no voters:
I20260807 06:42:49.567585 11514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:42:49.567749 11514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [term 1 LEADER]: Becoming Leader. State: Replica: 5a2dc5d6191043c9bef0f4da078d0c7f, State: Running, Role: LEADER
I20260807 06:42:49.567831 11514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } }
I20260807 06:42:49.568252 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a2dc5d6191043c9bef0f4da078d0c7f. Latest consensus state: current_term: 1 leader_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.568352 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:49.570132 11407 master_runner.cc:186] Error getting master registration for 127.11.114.253:38811: OK, instance_id { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" instance_seqno: 1786084969358925 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:42:49.572810 11407 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:42:49.573679 11531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.573765 11531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:49.577481 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.568887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54108 (local address 127.11.114.254:40649)
0807 06:42:49.569005 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.569007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.569020 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:42:49.569132 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.569133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.569145 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.569178 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.570615 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.571375 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.572401 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.572677 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.572722 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.574514 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.574515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.574515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.574523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.576097 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.576536 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.576537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.576538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.576598 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.576816 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.576817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.576818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.576963 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.577044 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.577283 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.577363 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 06:42:49.578279 11551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:42:49.578537 11551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 06:42:49.580169 11544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:42:49.580221 11544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:42:49.580336 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.569417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.11.114.252:36567)
0807 06:42:49.569451 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.569453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.569722 (+ 269us) server_negotiation.cc:407] Connection header received
0807 06:42:49.569836 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.569837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.569849 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.569881 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.573669 (+ 3788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.574439 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.575509 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.575844 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.575897 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.577282 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.577283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.577283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.577291 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.578953 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.579309 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.579310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.579310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.579373 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.579575 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.579576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.579576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.579719 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.579831 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.580143 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.580245 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:42:49.580626 11454 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [term 1 FOLLOWER]: Refusing update from remote peer 5a2dc5d6191043c9bef0f4da078d0c7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:49.580724 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.569078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.11.114.253:38811)
0807 06:42:49.569115 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.569117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.569446 (+ 329us) server_negotiation.cc:407] Connection header received
0807 06:42:49.569598 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.569599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.569621 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.569654 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.570910 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.571679 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.573603 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.573897 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.573955 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.577021 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.577023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.577023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.577044 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.579282 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.579797 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.579798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.579798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.579861 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.580114 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.580115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.580116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.580251 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.580323 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.580548 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.580632 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:42:49.585677 11545 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:49.587616 11531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a2dc5d6191043c9bef0f4da078d0c7f. Latest consensus state: current_term: 1 leader_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.587726 11531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:49.591749 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.591850 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:49.591802 11531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.591879 11531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:49.592100 11558 mvcc.cc:205] Tried to move back new op lower bound from 7315804035399045120 to 7315804035350503424. Current Snapshot: MvccSnapshot[applied={T|T < 7315804035399045120}]
I20260807 06:42:49.594383 11551 catalog_manager.cc:1347] Generated new cluster ID: 5baa5e5efc1f40f6abe17dc38f385a92
I20260807 06:42:49.594439 11551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:42:49.597571 11400 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } attempt: 3
Aug 07 06:42:49 dist-test-slave-gkw7 krb5kdc[11171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084969, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.114.253@KRBTEST.COM for kudu/127.11.114.252@KRBTEST.COM
I20260807 06:42:49.606962 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.597835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.114.252:36567 (local address 127.0.0.1:42826)
0807 06:42:49.597929 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 06:42:49.597941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:42:49.597980 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:42:49.598127 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:42:49.598128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:42:49.598318 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:42:49.598371 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.598373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.599363 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.599364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.600193 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:42:49.600196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.600783 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.600784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:42:49.600855 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.601325 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:42:49.601331 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:42:49.602925 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:42:49.605519 (+ 2594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.605520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.605520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.605782 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.606093 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:42:49.606094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:42:49.606094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:42:49.606170 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:42:49.606646 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:42:49.606647 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:42:49.606707 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 06:42:49.606756 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:42:49.606858 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 06:42:49.607923 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.597962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.11.114.252:36567)
0807 06:42:49.598014 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.598016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.598024 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:42:49.598054 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.598055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.598064 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.598119 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.598476 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.599265 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.600302 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.600648 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.600713 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.603007 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:42:49.603008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:42:49.603008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:42:49.603015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:42:49.605412 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.605879 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.605880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.605880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.606003 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:42:49.606259 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:42:49.606260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:42:49.606260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:42:49.606512 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:42:49.607509 (+ 997us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.607585 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.607768 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:42:49.609937 11400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e: Bootstrap starting.
I20260807 06:42:49.610704 11400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e: Neither blocks nor log segments found. Creating new log.
I20260807 06:42:49.610998 11400 log.cc:824] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:42:49.616314 11551 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:42:49.617103 11551 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:42:49.622318 11400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e: No bootstrap required, opened a new log
I20260807 06:42:49.624394 11400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } }
I20260807 06:42:49.624539 11400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ce0b647d3d4f658362c4835b0e243e, State: Initialized, Role: FOLLOWER
I20260807 06:42:49.624632 11400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } }
I20260807 06:42:49.624987 11400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:42:49.626669 11364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "69ce0b647d3d4f658362c4835b0e243e"
I20260807 06:42:49.626791 11364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:42:49.627709 11364 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a2dc5d6191043c9bef0f4da078d0c7f in term 1.
I20260807 06:42:49.628154 11363 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [term 1 FOLLOWER]: Refusing update from remote peer 5a2dc5d6191043c9bef0f4da078d0c7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:42:49.628429 11561 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:42:49.628799 11551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f: Generated new TSK 0
I20260807 06:42:49.628974 11551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:42:49.631575 11589 catalog_manager.cc:1259] Loaded cluster ID: 5baa5e5efc1f40f6abe17dc38f385a92
I20260807 06:42:49.631642 11589 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e: loading cluster ID for follower catalog manager: success
I20260807 06:42:49.632309 11574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.632478 11574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:49.632584 11574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a2dc5d6191043c9bef0f4da078d0c7f. Latest consensus state: current_term: 1 leader_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.632764 11574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:42:49.632877 11574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:49.632941 11589 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e: acquiring CA information for follower catalog manager: success
I20260807 06:42:49.633314 11589 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:42:49.633458 11574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ce0b647d3d4f658362c4835b0e243e [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:42:49.662758 11294 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.661104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.11.114.254:40649)
0807 06:42:49.661153 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.661155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.662065 (+ 910us) server_negotiation.cc:407] Connection header received
0807 06:42:49.662176 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.662177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.662189 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.662220 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.662690 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54120: BlockingRecv error: recv got EOF from 127.0.0.1:54120 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 06:42:49.663480 11404 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.661505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.11.114.253:38811)
0807 06:42:49.661572 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.661574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.662992 (+ 1418us) server_negotiation.cc:407] Connection header received
0807 06:42:49.663085 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.663086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.663098 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.663127 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.663421 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51416: BlockingRecv error: recv got EOF from 127.0.0.1:51416 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 06:42:49.664206 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.661881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42838 (local address 127.11.114.252:36567)
0807 06:42:49.661926 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.661928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.663711 (+ 1783us) server_negotiation.cc:407] Connection header received
0807 06:42:49.663802 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.663803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.663815 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.663843 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.664125 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42838: BlockingRecv error: recv got EOF from 127.0.0.1:42838 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:42:49.673676 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.669986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.11.114.253:38811)
0807 06:42:49.670025 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.670027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.670188 (+ 161us) server_negotiation.cc:407] Connection header received
0807 06:42:49.670300 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.670301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.670349 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.670389 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.670619 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.671359 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.672510 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.672812 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.672859 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.673157 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.673283 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.673338 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.673464 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.673485 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 06:42:49.676986 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.676151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.11.114.252:36567)
0807 06:42:49.676190 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.676192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.676416 (+ 224us) server_negotiation.cc:407] Connection header received
0807 06:42:49.676522 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.676523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.676542 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.676575 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.676935 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42840: BlockingRecv error: recv got EOF from 127.0.0.1:42840 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:49.682427 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.679258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.11.114.253:38811)
0807 06:42:49.679290 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.679292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.679529 (+ 237us) server_negotiation.cc:407] Connection header received
0807 06:42:49.679639 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.679640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.679649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.679676 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.679842 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.680533 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.681620 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.681911 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.681957 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.682169 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.682204 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.682232 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.682339 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.682359 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 06:42:49.685714 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.684817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42854 (local address 127.11.114.252:36567)
0807 06:42:49.684857 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.684859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.685118 (+ 259us) server_negotiation.cc:407] Connection header received
0807 06:42:49.685217 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.685218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.685230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.685260 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.685659 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42854: BlockingRecv error: recv got EOF from 127.0.0.1:42854 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:49.692003 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.688836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.11.114.253:38811)
0807 06:42:49.688872 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.688873 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.689107 (+ 234us) server_negotiation.cc:407] Connection header received
0807 06:42:49.689203 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.689204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.689214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.689241 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.689414 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.690135 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.691202 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.691474 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.691519 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.691745 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.691780 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.691810 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.691919 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.691936 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 06:42:49.695127 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.694270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42866 (local address 127.11.114.252:36567)
0807 06:42:49.694307 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.694310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.694563 (+ 253us) server_negotiation.cc:407] Connection header received
0807 06:42:49.694682 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.694683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.694695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.694727 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.695081 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42866: BlockingRecv error: recv got EOF from 127.0.0.1:42866 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:42:49.703872 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.700329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51452 (local address 127.11.114.253:38811)
0807 06:42:49.700365 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.700367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.700629 (+ 262us) server_negotiation.cc:407] Connection header received
0807 06:42:49.700727 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.700728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.700737 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.700765 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.700930 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.701826 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.703052 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.703333 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.703378 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.703597 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.703633 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.703663 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.703775 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.703794 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 06:42:49.707038 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.706153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42880 (local address 127.11.114.252:36567)
0807 06:42:49.706193 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.706194 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.706458 (+ 264us) server_negotiation.cc:407] Connection header received
0807 06:42:49.706561 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.706562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.706574 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.706620 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.706984 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42880: BlockingRecv error: recv got EOF from 127.0.0.1:42880 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:49.719667 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.716263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.11.114.253:38811)
0807 06:42:49.716311 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.716313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.716575 (+ 262us) server_negotiation.cc:407] Connection header received
0807 06:42:49.716695 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.716696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.716706 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.716736 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.716903 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.717732 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.718854 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.719158 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.719198 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.719398 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.719433 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.719459 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.719559 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.719577 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 06:42:49.722901 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.722042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.11.114.252:36567)
0807 06:42:49.722083 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.722084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.722329 (+ 245us) server_negotiation.cc:407] Connection header received
0807 06:42:49.722424 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.722425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.722436 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.722465 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.722849 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42890: BlockingRecv error: recv got EOF from 127.0.0.1:42890 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:49.743897 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.740454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.11.114.253:38811)
0807 06:42:49.740500 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.740503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.740807 (+ 304us) server_negotiation.cc:407] Connection header received
0807 06:42:49.740915 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.740916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.740927 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.740958 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.741151 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.741977 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.743108 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.743407 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.743446 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.743646 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.743679 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.743704 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.743797 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.743815 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 06:42:49.747072 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.746218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.11.114.252:36567)
0807 06:42:49.746262 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.746264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.746511 (+ 247us) server_negotiation.cc:407] Connection header received
0807 06:42:49.746616 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.746617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.746629 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.746661 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.747017 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42896: BlockingRecv error: recv got EOF from 127.0.0.1:42896 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:49.782998 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.780414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51480 (local address 127.11.114.253:38811)
0807 06:42:49.780461 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.780464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.780628 (+ 164us) server_negotiation.cc:407] Connection header received
0807 06:42:49.780702 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.780704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.780715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.780798 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.780943 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.781453 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.782293 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.782487 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.782567 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.782728 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.782766 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.782827 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.782881 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.782902 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 06:42:49.785548 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.784909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.11.114.252:36567)
0807 06:42:49.784954 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.784956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.785027 (+ 71us) server_negotiation.cc:407] Connection header received
0807 06:42:49.785092 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.785093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.785106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.785157 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.785475 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42910: BlockingRecv error: recv got EOF from 127.0.0.1:42910 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:42:49.853559 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.850888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.11.114.253:38811)
0807 06:42:49.850986 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.850989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.851074 (+ 85us) server_negotiation.cc:407] Connection header received
0807 06:42:49.851147 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.851148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.851162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.851253 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.851408 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.851918 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.852761 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.852963 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.853080 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.853239 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.853282 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.853350 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.853418 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.853439 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 06:42:49.856175 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.855505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.11.114.252:36567)
0807 06:42:49.855581 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.855584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.855593 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:42:49.855682 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.855683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.855698 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.855756 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.856115 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42916: BlockingRecv error: recv got EOF from 127.0.0.1:42916 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:42:49.987890 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:49.985510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.11.114.253:38811)
0807 06:42:49.985624 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.985626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.985702 (+ 76us) server_negotiation.cc:407] Connection header received
0807 06:42:49.985778 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.985779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.985788 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.985835 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:49.985967 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.986469 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.987269 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:49.987457 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:49.987514 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:49.987653 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:49.987682 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:49.987731 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:49.987813 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 06:42:49.987829 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 06:42:49.990379 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:49.989740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.11.114.252:36567)
0807 06:42:49.989827 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:49.989829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:49.989878 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:42:49.989934 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:49.989935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:49.989945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:49.989990 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:49.990311 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42920: BlockingRecv error: recv got EOF from 127.0.0.1:42920 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:42:50.211153 11744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a2dc5d6191043c9bef0f4da078d0c7f" member_type: VOTER last_known_addr { host: "127.11.114.254" port: 40649 } } peers { permanent_uuid: "69ce0b647d3d4f658362c4835b0e243e" member_type: VOTER last_known_addr { host: "127.11.114.253" port: 38811 } } peers { permanent_uuid: "114acace0b06464ebbf109656386cf98" member_type: VOTER last_known_addr { host: "127.11.114.252" port: 36567 } } }
I20260807 06:42:50.211346 11744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a2dc5d6191043c9bef0f4da078d0c7f [sys.catalog]: This master's current role is: LEADER
I20260807 06:42:50.250075 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:50.247601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51500 (local address 127.11.114.253:38811)
0807 06:42:50.247689 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:50.247691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:50.247749 (+ 58us) server_negotiation.cc:407] Connection header received
0807 06:42:50.247802 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:50.247803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:50.247812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:50.247876 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:50.248017 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.248530 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.249341 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.249595 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.249653 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:50.249820 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:50.249851 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:50.249889 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:50.249988 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 06:42:50.250006 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 06:42:50.252404 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:42:50.251873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.11.114.252:36567)
0807 06:42:50.251954 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:50.251956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:50.251964 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:42:50.251986 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:50.251987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:50.251996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:50.252045 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:42:50.252364 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42924: BlockingRecv error: recv got EOF from 127.0.0.1:42924 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:42:50.580698 11544 catalog_manager.cc:1259] Loaded cluster ID: 5baa5e5efc1f40f6abe17dc38f385a92
I20260807 06:42:50.580782 11544 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98: loading cluster ID for follower catalog manager: success
I20260807 06:42:50.581681 11544 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98: acquiring CA information for follower catalog manager: success
I20260807 06:42:50.582069 11544 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 114acace0b06464ebbf109656386cf98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:42:50.768203 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:50.765612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.11.114.253:38811)
0807 06:42:50.765746 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:50.765749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:50.765782 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:42:50.765856 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:50.765857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:50.765868 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:50.765998 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:50.766180 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.766801 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.767599 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.767794 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.767848 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:50.767978 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:50.768007 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:50.768045 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:50.768104 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 06:42:50.768121 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
I20260807 06:42:50.772150 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:50.769960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42932 (local address 127.11.114.252:36567)
0807 06:42:50.770078 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:50.770080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:50.770088 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:42:50.770111 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:50.770112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:50.770166 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:50.770234 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:50.770332 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.770798 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.771450 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.771640 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.771689 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:50.771817 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:50.771922 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:50.771979 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:50.772013 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 06:42:50.772030 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260807 06:42:50.776868 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:50.774297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.11.114.252:36567)
0807 06:42:50.774380 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:50.774382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:50.774691 (+ 309us) server_negotiation.cc:407] Connection header received
0807 06:42:50.774725 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:50.774726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:50.774735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:50.774790 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:50.774890 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.775355 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.776168 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.776370 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.776444 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:50.776619 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:50.776660 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:50.776710 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:50.776770 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 06:42:50.776790 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 06:42:50.777484 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:50.774200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.11.114.253:38811)
0807 06:42:50.774237 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:50.774239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:50.774356 (+ 117us) server_negotiation.cc:407] Connection header received
0807 06:42:50.774506 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:50.774507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:50.774519 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:50.774575 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:50.774733 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.775440 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.776561 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.776875 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.776935 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:50.777209 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:50.777242 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:50.777269 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:50.777366 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 06:42:50.777387 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:42:50.777719 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:42:50.774047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54128 (local address 127.11.114.254:40649)
0807 06:42:50.774180 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 06:42:50.774183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:42:50.774255 (+ 72us) server_negotiation.cc:407] Connection header received
0807 06:42:50.774308 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:42:50.774309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:42:50.774364 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:42:50.774453 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:42:50.774610 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.775396 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.776512 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:42:50.776828 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:42:50.776930 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:42:50.777109 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:42:50.777243 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:42:50.777306 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:42:50.777614 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 06:42:50.777641 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":55,"threads_started":1}
I20260807 06:42:50.779559 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 11190
I20260807 06:42:50.796741 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 11299
I20260807 06:42:50.804471 11723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZbZY4B/build/release/bin/kudu with pid 11407
2026-08-07T06:42:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3074 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6235 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (286459 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST